-- Logs begin at Mon 2022-10-03 14:11:28 UTC, end at Mon 2022-10-03 15:16:10 UTC. -- Oct 03 14:37:26.696400 dv-cinder-859877-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 03 14:37:26.706578 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 3 14:37:26 2022] *** Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: nodename: dv-cinder-859877-2-iscsi Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: machine: x86_64 Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: clock source: unix Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: pcre jit disabled Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: detected number of CPU cores: 4 Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: current working directory: / Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: detected binary path: /usr/bin/uwsgi-core Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: your processes number limit is 47680 Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: your memory page size is 4096 bytes Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: detected max file descriptor number: 2048 Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: lock engine: pthread robust mutexes Oct 03 14:37:26.707190 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: thunder lock: enabled Oct 03 14:37:26.708955 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 03 14:37:26.708955 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 03 14:37:26.746247 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: Python main interpreter initialized at 0x55aeb26189c0 Oct 03 14:37:26.746247 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: python threads support enabled Oct 03 14:37:26.746247 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: your server socket listen backlog is limited to 100 connections Oct 03 14:37:26.746247 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: your mercy for graceful operations on workers is 90 seconds Oct 03 14:37:26.746602 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: mapped 403077 bytes (393 KB) for 2 cores Oct 03 14:37:26.746683 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: *** Operational MODE: preforking *** Oct 03 14:37:26.746928 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: *** uWSGI is running in multiple interpreter mode *** Oct 03 14:37:26.746928 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: spawned uWSGI master process (pid: 87821) Oct 03 14:37:26.747117 dv-cinder-859877-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 03 14:37:26.748455 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: spawned uWSGI worker 1 (pid: 87830, cores: 1) Oct 03 14:37:26.748455 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: spawned uWSGI worker 2 (pid: 87831, cores: 1) Oct 03 14:37:26.748455 dv-cinder-859877-2-iscsi devstack@keystone.service[87821]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 03 14:37:31.038653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 14:37:31.038653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: from cryptography.utils import int_from_bytes Oct 03 14:37:31.039430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 14:37:31.039430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: from cryptography.utils import int_from_bytes Oct 03 14:37:31.039844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 14:37:31.039844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: from cryptography.utils import int_from_bytes Oct 03 14:37:31.039945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 14:37:31.039945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: from cryptography.utils import int_from_bytes Oct 03 14:37:31.089037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 03 14:37:31.089231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 03 14:37:31.089427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] command line args: [] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 03 14:37:31.089726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 03 14:37:31.089871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 03 14:37:31.089951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ================================================================================ {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 03 14:37:31.090064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 03 14:37:31.090152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] admin_token = **** {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.090253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] command line args: [] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 03 14:37:31.090388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] config_dir = [] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.090515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 03 14:37:31.090597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.090675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ================================================================================ {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 03 14:37:31.090751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] config_source = [] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.090913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] debug = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.090996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] admin_token = **** {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.091117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.091371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] config_dir = [] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.091371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.091484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.091540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] config_source = [] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.091683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] debug = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.091784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.091890 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.092022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.092125 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.092220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.092312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.092412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] list_limit = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.092512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.092612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] log_config_append = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.092713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.092813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.092914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] log_dir = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.093017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] list_limit = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.093112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] log_file = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.093211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] log_config_append = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.093316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.093440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.093536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] log_dir = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.093625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.093696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] log_file = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.093769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.093838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.094014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.094108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.094185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.094285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.094368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.094527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.094605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.094702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.094827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.094905 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.095066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.095185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.095311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.095423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.095542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.095660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.095779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.095892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.096018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.096138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.096252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.096379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.096490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.096603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.096717 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.096898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.097010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.097123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] publish_errors = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.097351 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.097524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.097653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] publish_errors = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.097772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.097772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.098000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.098000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.098221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.098339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.098470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.098594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.098708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] standard_threads = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.098827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.098947 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.099067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] standard_threads = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.099193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.099308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.099421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.099421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] use_journal = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.099639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.099753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] use_json = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.099915 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.100032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] use_stderr = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.100151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] use_journal = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.100272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] use_syslog = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.100392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] use_json = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.100392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] use_stderr = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.100612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.100732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] use_syslog = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.100852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.100966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 14:37:31.101085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.101200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.101324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.101439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.101589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.101723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.101837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.101956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.102079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.102199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.102320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.102439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.102553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.102671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.102796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.102911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.103024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.103142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.103256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.103359 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.103479 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.103593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.103705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.103825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.103938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.104056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.104182 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.104297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.104418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.104532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.104644 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.104769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.104890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.105003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.105116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.105229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.105341 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.105558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.105670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.105790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.105790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.106002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.106127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.106250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.106363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.106484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.106597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.106717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.106830 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.106953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.107067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.107179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.107291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.107404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.107518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.107639 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.107759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.107873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.107986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.108099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.108211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.108332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.108452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.108567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.108686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.108799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.108912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.109033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.109153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.109267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.109380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.109510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.109628 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.109753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.109867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.109980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.110092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.110211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.110331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.110454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.110568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.110682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.110800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.110914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.111027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.111215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.111332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.111445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.111557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.111676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.111804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.111931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.112045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.112169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.112284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.112404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.112518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.112622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.112718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.112813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.112906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.113007 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.113110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.113207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.113307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.113403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.113522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.113625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.113753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.113869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.113987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.114101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.114247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.114374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.114497 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.114613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.114726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.114840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.114959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.114959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.115184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.115299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.115412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.115524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.115646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.115759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.115882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.116010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.116144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.116261 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.116375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.116488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.116601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.116720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.116843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.116954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.117068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.117182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.117284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.117396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.117574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.117694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.117807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.117920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.118034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.118148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.118532 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.118652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.118766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.118880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.118993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.119107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.119229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.119343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.119458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.119571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.119684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.119798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.119919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.120034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.120149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.120263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.120377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.120490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.120612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.120717 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.120829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.120942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.121062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.121301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.121416 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.121543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.121658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.121782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.121898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.122020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.122135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.122257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.122372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.122491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.122606 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.122795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.122912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.123026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.123139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.123253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.123376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.123499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.123623 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.124230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.124350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.124465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.124578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.124700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.124827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.124827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.125043 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.125043 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.125256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.125256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.125486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.125618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.125744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.125866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.125985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.126106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.126233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.126349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.126463 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.126582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.126697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.126837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] auth.external = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.126962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.127079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.127202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] auth.external = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.127202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.127420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.127420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.127636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.127757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] auth.password = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.127882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.127998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] auth.token = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.128113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] auth.password = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.128226 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.128341 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] auth.token = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.128455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.128578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.128694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.128809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.128922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.129037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.129151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.129273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.129390 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.129536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.129665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.129782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.129896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] credential.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.130019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.130134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.130255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] credential.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.130371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.130486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.130601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.130723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.130840 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.130955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.131069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.131183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.131298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.131420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.131544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.131661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.131776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.131890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.132004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.132127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.132243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.132358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.132473 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.132589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.132712 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.132834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.132950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.133071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.133193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.133309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.133431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.133565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.133682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] federation.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.133798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.133913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.134027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] federation.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.134156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.134274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.134388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.134388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.134603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.134724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.134849 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.134965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.135088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.135210 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.135327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.135443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.135570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.135687 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.135804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.135919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.136039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.136161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.136285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.136402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.136522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.136638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.136753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.136869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.137062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.137181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.137303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.137418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.137546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.137663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.137790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.137908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.138023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.138138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.138253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.138367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.138490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.138614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.138732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.138846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.138962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.139076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.139199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.139315 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.139484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.140096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.140247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.140364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.140488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.140611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.140728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.140848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.140964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.141080 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.141204 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.141320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.141435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.141564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.141681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.141809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.141925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.142040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.142156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.142270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.142270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.142493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.142610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.142724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.142838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.142953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.143067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.143191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.143307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.143422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.143536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.143650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.143764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.143886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.144003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.144117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.144233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.144355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.144471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.144594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.144715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.144831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.144946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.145060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.145175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.145299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.145415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.145584 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.145701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.145816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.145931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.146059 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.146177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.146292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.146407 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.146521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.146634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.146757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.146873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.146987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.146987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.147213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.147213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.147437 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.147554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.147669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.147783 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.147897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.148011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.148134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.148251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.148365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.148481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.148596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.148710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.148833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.148950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.149064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.149179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.149293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.149407 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.149552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.149708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.149826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.149940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.150055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.150169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.150292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.150415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.150532 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.150646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.150760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.150875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.150997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.151113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.151113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.151336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.151451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.151566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.151730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.151850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.151965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.152080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.152195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.152309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.152432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.152549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.152665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.152780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.152894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.153008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.153131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.153247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.153362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.153483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.153664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.153780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.153907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.154024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.154140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.154255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.154369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.154484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.154606 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.154724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.154845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.154961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.155100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.155216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.155339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.155456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.155570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.155685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.155800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.155914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.156033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.156157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.156273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.156387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.156501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.156615 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.156738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.156855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.156976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.157092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.157206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.157329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.157446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.157577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.157692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] resource.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.157807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.157922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.158048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.158165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.158280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.158394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.158509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.158624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.158746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] resource.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.158863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.158979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.159094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.159208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.159322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.159445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.159562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.159677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.159791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.159906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.160022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] role.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.160022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] role.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.160310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.160426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.160541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.160655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.160770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.160893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.161012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.161126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.161241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] role.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.161356 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.161470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] role.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.161616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.161733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.161849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.161963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.162078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.162193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.162319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.162437 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.162552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.162667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.162781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.162896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.163022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.163140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.163255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.163369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.163484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.163599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.163599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.163824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.163940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.164053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.164169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.164290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.164414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.164530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.164645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.164759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.164759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.164982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.165099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.165216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.165330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.165445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.165610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.165743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.165861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.165977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.166095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.166211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.166325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.166448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.166565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.166680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.166794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.166909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.167024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.167147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.167270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.167386 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.167501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.167615 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.167730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.167852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.167968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.168083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.168197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] token.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.168197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.168412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.168535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.168652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.168766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.168881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.168995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.169111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.169233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.169350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.169465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.169600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] token.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.169715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.169829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.169956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.170073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.170188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.170303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.170419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.170534 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.171170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.171289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.171388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.171504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.171619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.171735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.171858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.171975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.171975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.172191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.172313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.172438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.172557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.172680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.172797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.172912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.connection = **** {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.173027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.173141 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.173256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.173379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.173513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.173644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.173770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.173887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.173887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.174102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.174216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.174339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.connection = **** {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.174455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.174570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.174686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.174801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.174916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.175038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.175154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.175269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.175384 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.175499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.175613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.175737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.175853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.175968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.176083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.176190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.176303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.176426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.176545 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.176661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.176776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.176892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.177006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.177128 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.177245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 03 14:37:31.177360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.177360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.177360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.177360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.177360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.177360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 14:37:31.177360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87830) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 03 14:37:31.204282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.204510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.204730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.204814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.204991 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.205224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.213063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.213551 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.241306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.241561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.243032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.243521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.294978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.295492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.343963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.344196 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.345266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.345524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.492505 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.492737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.492967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.493192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.493696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.493918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.496541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.496774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.496993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.497215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.497439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.497673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.517684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.517888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.517974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.518123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.518205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.518343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.518424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.518560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.518651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.518773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.518854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.518986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.519124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.519212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.530683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.530984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.543294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.543566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.543755 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.543839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.543979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.544061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.544198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.544281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.544415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.544498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.544631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87831) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.544713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.544902 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.545123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87830) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 14:37:31.561911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.562115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 14:37:31.562456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.562537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.562652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 14:37:31.562750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 14:37:31.562924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.563006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.563183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.563264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 14:37:31.563437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.563520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.563690 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.563771 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.564222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.564489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.573446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.573752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.573983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.574065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.574269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.574363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.574550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.574842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.581857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.582425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.584745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.585025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.585303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.585383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.585603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.585687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.585885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.585967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.586144 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.586225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.586409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.586499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.586679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.586759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.586945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.587025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.587213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.587295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.587476 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.587595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.587877 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.588152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.610101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.611552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.611836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.612514 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.614021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.614314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.616502 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.616781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.617058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.619271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.619560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.619846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.623638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.624451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.626810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.627107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.630076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.630372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.630657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.630943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.631217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.631307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.631489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.631610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.631754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.631901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.632020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.632191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.632479 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.632770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.633044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.633330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.647949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.648236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.649621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.649907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.652059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.653804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.655318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.655612 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.657059 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.657387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.659829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.661699 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.662956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.664869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.666189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.666486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.666767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.667047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.667331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.667613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.668143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.668450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.668743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.669035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.669322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.669645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.683468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.683753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.684019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.684280 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.685679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.685975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.686787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.687069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.691662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.693192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.694060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.694737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.695011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.695618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.697160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.697441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.697708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.697990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.698277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.698562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.698944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.700132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.700431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.700828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.701119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.701415 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.710111 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.710391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.710665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.710949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.711327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.711620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.711898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.712783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.713073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.713459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.713770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.714065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.714348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.714634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.727789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.730685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.730771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.733643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.733725 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.735073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.736600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.736692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.736943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.737338 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.738103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.739702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.739991 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.740289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.744017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.745462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.747050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.748348 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.748521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.749249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.749560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.749839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.752949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.753245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.753551 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.753835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.757622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.757912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.758191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.761752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.762074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.762361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.765086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87831) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.765379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.765691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.765970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.766245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.766515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.766788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.767073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.767343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.767610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.767895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.768168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.768429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.768684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87831) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.769381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87830) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 14:37:31.769698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.769985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.770265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.770541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.770818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.771096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.771385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.771688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.771962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.772247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.772513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.772782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.773057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87830) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 14:37:31.803131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55aeb26189c0 pid: 87831 (default app) Oct 03 14:37:31.806309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0f532-163b-4f1b-94b0-b83ea737b4fa None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:31.806595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0f532-163b-4f1b-94b0-b83ea737b4fa None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:31.806676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55aeb26189c0 pid: 87830 (default app) Oct 03 14:37:31.806865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0f532-163b-4f1b-94b0-b83ea737b4fa None None] PATH_INFO: `/v3` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:31.810986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Mon Oct 3 14:37:31 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:37:38.567431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c044d-fb11-469e-8810-88cdbe3e0d58 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:38.567925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c044d-fb11-469e-8810-88cdbe3e0d58 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:38.567925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c044d-fb11-469e-8810-88cdbe3e0d58 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:38.572186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:37:38.578076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:38.578596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:38.578596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:38.648128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: INFO dbcounter [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] Registered counter for database keystone Oct 03 14:37:38.666615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG oslo_db.sqlalchemy.engines [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87831) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 03 14:37:38.673299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writer thread running {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 03 14:37:38.852054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG passlib.handlers.bcrypt [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87831) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 03 14:37:38.852357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG passlib.handlers.bcrypt [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87831) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 03 14:37:38.910380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:38.911998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:39.039688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 03 14:37:39.504727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c044d-fb11-469e-8810-88cdbe3e0d58 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:39.574432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: INFO dbcounter [None req-e34c044d-fb11-469e-8810-88cdbe3e0d58 None None] Registered counter for database keystone Oct 03 14:37:39.599672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG oslo_db.sqlalchemy.engines [None req-e34c044d-fb11-469e-8810-88cdbe3e0d58 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87830) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 03 14:37:39.605805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writer thread running {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 03 14:37:39.725309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c044d-fb11-469e-8810-88cdbe3e0d58 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:39.729557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc724d5-6fa0-4106-9dd0-c2feb8a52cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:39.730188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc724d5-6fa0-4106-9dd0-c2feb8a52cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:39.730470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc724d5-6fa0-4106-9dd0-c2feb8a52cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:39.730741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc724d5-6fa0-4106-9dd0-c2feb8a52cf9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:39.739367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc724d5-6fa0-4106-9dd0-c2feb8a52cf9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:39.790740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0dc724d5-6fa0-4106-9dd0-c2feb8a52cf9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 03 14:37:39.791085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc724d5-6fa0-4106-9dd0-c2feb8a52cf9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:39.800721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Mon Oct 3 14:37:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 03 14:37:39.806545 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:39.823647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860f30a-22a3-40f8-a918-f3e9df770b9d None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:39.828082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35245655-651f-4ee3-b403-60928f227e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:39.829120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35245655-651f-4ee3-b403-60928f227e0f admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:39.829398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35245655-651f-4ee3-b403-60928f227e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:39.829700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35245655-651f-4ee3-b403-60928f227e0f admin admin] PATH_INFO: `/v3/projects/47484d92809b4b9a846447c512be0a94` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:39.831262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35245655-651f-4ee3-b403-60928f227e0f admin admin] RBAC: Authorizing `identity:get_project(project_id=47484d92809b4b9a846447c512be0a94)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:39.883765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WARNING keystone.common.rbac_enforcer.enforcer [None req-35245655-651f-4ee3-b403-60928f227e0f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 03 14:37:39.884110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35245655-651f-4ee3-b403-60928f227e0f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:39.888529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Mon Oct 3 14:37:39 2022] GET /identity/v3/projects/47484d92809b4b9a846447c512be0a94 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:37:41.999812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d4b9-3935-4ec7-9a6d-daa7f590952d None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:42.000376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d4b9-3935-4ec7-9a6d-daa7f590952d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:42.000376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d4b9-3935-4ec7-9a6d-daa7f590952d None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:42.004439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:42.009824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e9385-032c-48bc-b623-690a6dffa4d9 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:42.010233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e9385-032c-48bc-b623-690a6dffa4d9 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:42.010379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e9385-032c-48bc-b623-690a6dffa4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:42.031282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-9e1e9385-032c-48bc-b623-690a6dffa4d9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:42.032801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9e1e9385-032c-48bc-b623-690a6dffa4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:42.040539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 03 14:37:42.302649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9d4b9-3935-4ec7-9a6d-daa7f590952d None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:42.314569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9d4b9-3935-4ec7-9a6d-daa7f590952d None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:42.318710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5598275-4ea8-4712-b46e-a1e0a04fd13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:42.319300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598275-4ea8-4712-b46e-a1e0a04fd13a admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:42.319583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598275-4ea8-4712-b46e-a1e0a04fd13a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:42.319853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598275-4ea8-4712-b46e-a1e0a04fd13a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:42.327312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5598275-4ea8-4712-b46e-a1e0a04fd13a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:42.329075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5598275-4ea8-4712-b46e-a1e0a04fd13a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:42.337860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Mon Oct 3 14:37:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 14:37:42.343419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e9385-032c-48bc-b623-690a6dffa4d9 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:42.352423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e9385-032c-48bc-b623-690a6dffa4d9 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:42.356712 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0a8c4-0224-4bfb-b6cf-86e9aab683ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:42.357907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0a8c4-0224-4bfb-b6cf-86e9aab683ec admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:42.358197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0a8c4-0224-4bfb-b6cf-86e9aab683ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:42.358468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0a8c4-0224-4bfb-b6cf-86e9aab683ec admin admin] PATH_INFO: `/v3/users/dd9397aad94548f284667a53c0b499d2` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:42.359783 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0a8c4-0224-4bfb-b6cf-86e9aab683ec admin admin] RBAC: Authorizing `identity:get_user(user_id=dd9397aad94548f284667a53c0b499d2)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:42.361674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0a8c4-0224-4bfb-b6cf-86e9aab683ec admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:42.363170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Mon Oct 3 14:37:42 2022] GET /identity/v3/users/dd9397aad94548f284667a53c0b499d2 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:37:44.535325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f172a48-51a4-41a0-8fc6-48f1a5ff651d None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:44.535844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f172a48-51a4-41a0-8fc6-48f1a5ff651d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:44.535844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f172a48-51a4-41a0-8fc6-48f1a5ff651d None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:44.537182 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:44.542252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2e9d9-37a6-4df3-8de7-99b0a26ddd2e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:44.542621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2e9d9-37a6-4df3-8de7-99b0a26ddd2e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:44.542795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2e9d9-37a6-4df3-8de7-99b0a26ddd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:44.561000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-99e2e9d9-37a6-4df3-8de7-99b0a26ddd2e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:44.562147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-99e2e9d9-37a6-4df3-8de7-99b0a26ddd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:44.590166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340339-4454-474a-b5c5-867f8c307c6b None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:44.590552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340339-4454-474a-b5c5-867f8c307c6b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:44.590831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340339-4454-474a-b5c5-867f8c307c6b None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:44.592152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:37:44.597229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a416-38e7-46b2-a576-e8d80dca7963 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:44.597631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a416-38e7-46b2-a576-e8d80dca7963 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:44.597906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a416-38e7-46b2-a576-e8d80dca7963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:44.639549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG passlib.handlers.bcrypt [None req-18c8a416-38e7-46b2-a576-e8d80dca7963 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87830) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 03 14:37:44.639935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG passlib.handlers.bcrypt [None req-18c8a416-38e7-46b2-a576-e8d80dca7963 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87830) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 03 14:37:44.642386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:37:44.689210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-18c8a416-38e7-46b2-a576-e8d80dca7963 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:44.690807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-18c8a416-38e7-46b2-a576-e8d80dca7963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:44.786832 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:37:44.908657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2e9d9-37a6-4df3-8de7-99b0a26ddd2e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:44.932887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:45.012622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2e9d9-37a6-4df3-8de7-99b0a26ddd2e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:45.045974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893510f6-5ac8-4f1f-a18b-c04d37aafb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:45.046606 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893510f6-5ac8-4f1f-a18b-c04d37aafb7d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:45.046882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893510f6-5ac8-4f1f-a18b-c04d37aafb7d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:45.047208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893510f6-5ac8-4f1f-a18b-c04d37aafb7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:45.048459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8a416-38e7-46b2-a576-e8d80dca7963 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:45.058327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893510f6-5ac8-4f1f-a18b-c04d37aafb7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-893510f6-5ac8-4f1f-a18b-c04d37aafb7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:37:45.067764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:37:45.068805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:37:45.070103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:37:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:37:45.074522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893510f6-5ac8-4f1f-a18b-c04d37aafb7d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:45.148468 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8a416-38e7-46b2-a576-e8d80dca7963 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:45.151362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:45.153918 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:45.172731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893510f6-5ac8-4f1f-a18b-c04d37aafb7d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:45.186441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067de1b-3742-4df7-8bb9-9fe51fd69a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:45.187304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067de1b-3742-4df7-8bb9-9fe51fd69a04 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:45.187583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067de1b-3742-4df7-8bb9-9fe51fd69a04 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:45.187855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067de1b-3742-4df7-8bb9-9fe51fd69a04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:45.206109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ef630-0452-48d3-89ae-c1b0140567da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:45.206711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ef630-0452-48d3-89ae-c1b0140567da None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:45.206989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ef630-0452-48d3-89ae-c1b0140567da None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:45.207261 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ef630-0452-48d3-89ae-c1b0140567da None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:45.207927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ef630-0452-48d3-89ae-c1b0140567da None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:37:45.208229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ef630-0452-48d3-89ae-c1b0140567da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:45.209647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ef630-0452-48d3-89ae-c1b0140567da None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:45.221037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:37:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:37:45.226630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ef630-0452-48d3-89ae-c1b0140567da None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:45.253998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067de1b-3742-4df7-8bb9-9fe51fd69a04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:45.255535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067de1b-3742-4df7-8bb9-9fe51fd69a04 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:45.325548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ef630-0452-48d3-89ae-c1b0140567da None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:45.351423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 8/16] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 14:37:45.357037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067de1b-3742-4df7-8bb9-9fe51fd69a04 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:45.359578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3c9d1-9732-417c-b769-2d48ce1bd4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:45.360211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3c9d1-9732-417c-b769-2d48ce1bd4df None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:45.360335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:45.360480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3c9d1-9732-417c-b769-2d48ce1bd4df None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:45.361206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3c9d1-9732-417c-b769-2d48ce1bd4df None admin] PATH_INFO: `/v3/users/dd9397aad94548f284667a53c0b499d2` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:45.362395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3c9d1-9732-417c-b769-2d48ce1bd4df None admin] RBAC: Authorizing `identity:get_user(user_id=dd9397aad94548f284667a53c0b499d2)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:45.363879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3c9d1-9732-417c-b769-2d48ce1bd4df None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:45.365430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 9/17] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:37:45 2022] GET /identity/v3/users/dd9397aad94548f284667a53c0b499d2 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:37:45.371001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3c9d1-9732-417c-b769-2d48ce1bd4df None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:45.385680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:45.454347 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067de1b-3742-4df7-8bb9-9fe51fd69a04 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:45.468496 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3c9d1-9732-417c-b769-2d48ce1bd4df None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:45.487413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fab7c1-d9da-4aaf-9be1-d7226cf56e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:45.488272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fab7c1-d9da-4aaf-9be1-d7226cf56e67 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:45.488549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fab7c1-d9da-4aaf-9be1-d7226cf56e67 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:45.488817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fab7c1-d9da-4aaf-9be1-d7226cf56e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:45.501285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c12ce-f5cf-4ec8-b161-90e0db6eaf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:45.501936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c12ce-f5cf-4ec8-b161-90e0db6eaf72 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:45.502215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c12ce-f5cf-4ec8-b161-90e0db6eaf72 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:45.502492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c12ce-f5cf-4ec8-b161-90e0db6eaf72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:45.503596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c12ce-f5cf-4ec8-b161-90e0db6eaf72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:45.505038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c12ce-f5cf-4ec8-b161-90e0db6eaf72 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:45.506593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 10/18] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:37:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:37:45.508479 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fab7c1-d9da-4aaf-9be1-d7226cf56e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:45.510009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fab7c1-d9da-4aaf-9be1-d7226cf56e67 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:45.512200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c12ce-f5cf-4ec8-b161-90e0db6eaf72 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-b8fab7c1-d9da-4aaf-9be1-d7226cf56e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:37:45.528523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:37:45.529998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:37:45.530895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:37:45.530895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:37:45.530895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:37:45.530895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 9/19] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:37:45.535750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fab7c1-d9da-4aaf-9be1-d7226cf56e67 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:45.590020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:45.592077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:45.610048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c12ce-f5cf-4ec8-b161-90e0db6eaf72 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:45.639908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fab7c1-d9da-4aaf-9be1-d7226cf56e67 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:45.643097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06d029-8a3a-4ad2-a9a3-00396c738da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:45.643703 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06d029-8a3a-4ad2-a9a3-00396c738da7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:45.643978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06d029-8a3a-4ad2-a9a3-00396c738da7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:45.644244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06d029-8a3a-4ad2-a9a3-00396c738da7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:45.644963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06d029-8a3a-4ad2-a9a3-00396c738da7 None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.domain.id=default, user.id=dd9397aad94548f284667a53c0b499d2) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:37:45.645264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06d029-8a3a-4ad2-a9a3-00396c738da7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:45.646735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06d029-8a3a-4ad2-a9a3-00396c738da7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:45.653181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 11/20] 10.222.0.46 () {56 vars in 1322 bytes} [Mon Oct 3 14:37:45 2022] GET /identity/v3/role_assignments?user.id=dd9397aad94548f284667a53c0b499d2&scope.domain.id=default&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 233 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:37:45.673980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9df1da-2ab6-459c-86f2-0f023560d70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:45.674611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9df1da-2ab6-459c-86f2-0f023560d70a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:45.674888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9df1da-2ab6-459c-86f2-0f023560d70a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:45.675155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9df1da-2ab6-459c-86f2-0f023560d70a None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:45.675825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9df1da-2ab6-459c-86f2-0f023560d70a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:37:45.676127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9df1da-2ab6-459c-86f2-0f023560d70a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:45.677583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9df1da-2ab6-459c-86f2-0f023560d70a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:45.697092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 10/21] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:37:48.127738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e05049-4821-4f55-b5a3-3ed86ed10ba9 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:48.128708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e05049-4821-4f55-b5a3-3ed86ed10ba9 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:48.129195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e05049-4821-4f55-b5a3-3ed86ed10ba9 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:48.130773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 12/22] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:48.136058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f992-58a8-4bca-bf5d-40bad7bbe852 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:48.136855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f992-58a8-4bca-bf5d-40bad7bbe852 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:48.137320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f992-58a8-4bca-bf5d-40bad7bbe852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:48.166893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-25c0f992-58a8-4bca-bf5d-40bad7bbe852 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:48.168567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-25c0f992-58a8-4bca-bf5d-40bad7bbe852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:48.256023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 11/23] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:37:48.368908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa37a3-cead-48b0-8b23-61b24a9dd5f6 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:48.369571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa37a3-cead-48b0-8b23-61b24a9dd5f6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:48.370037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa37a3-cead-48b0-8b23-61b24a9dd5f6 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:48.371868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 13/24] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:48.377150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f658f-6dcf-47fd-980b-8cffbd239133 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:48.377700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f658f-6dcf-47fd-980b-8cffbd239133 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:48.378222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f658f-6dcf-47fd-980b-8cffbd239133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:48.402495 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9f7f658f-6dcf-47fd-980b-8cffbd239133 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:48.404572 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9f7f658f-6dcf-47fd-980b-8cffbd239133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:48.437231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6dfb8-c64d-4716-9a8d-c6d5bcb9fad1 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:48.437902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6dfb8-c64d-4716-9a8d-c6d5bcb9fad1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:48.438427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6dfb8-c64d-4716-9a8d-c6d5bcb9fad1 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:48.440331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 14/25] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:37:48.445642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fc74f-ecc5-4ca6-a75b-36387f5c86e2 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:48.446233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fc74f-ecc5-4ca6-a75b-36387f5c86e2 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:48.446715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fc74f-ecc5-4ca6-a75b-36387f5c86e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:48.457430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:48.478832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-6a4fc74f-ecc5-4ca6-a75b-36387f5c86e2 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:48.480375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6a4fc74f-ecc5-4ca6-a75b-36387f5c86e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:48.483499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 12/26] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:37:48.570239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 15/27] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:37:48.722977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f658f-6dcf-47fd-980b-8cffbd239133 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:48.853781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f658f-6dcf-47fd-980b-8cffbd239133 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:48.901542 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fc74f-ecc5-4ca6-a75b-36387f5c86e2 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:48.903531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16692a85-d1d3-49e7-b6ed-0b77ec8fd464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:48.904718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16692a85-d1d3-49e7-b6ed-0b77ec8fd464 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:48.905004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16692a85-d1d3-49e7-b6ed-0b77ec8fd464 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:48.905268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16692a85-d1d3-49e7-b6ed-0b77ec8fd464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:48.923360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16692a85-d1d3-49e7-b6ed-0b77ec8fd464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:48.924867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16692a85-d1d3-49e7-b6ed-0b77ec8fd464 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-16692a85-d1d3-49e7-b6ed-0b77ec8fd464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:37:48.949369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:37:48.950289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:37:48.951115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:37:48.951115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:37:48.951115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:37:48.951115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 13/28] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:37:48.958994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16692a85-d1d3-49e7-b6ed-0b77ec8fd464 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:49.032211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:49.032857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:49.086170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fc74f-ecc5-4ca6-a75b-36387f5c86e2 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:49.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e469b31-31ff-431b-97af-8415bd609473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:49.136640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e469b31-31ff-431b-97af-8415bd609473 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:49.137064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e469b31-31ff-431b-97af-8415bd609473 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:49.137540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e469b31-31ff-431b-97af-8415bd609473 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:49.159822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e469b31-31ff-431b-97af-8415bd609473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-9e469b31-31ff-431b-97af-8415bd609473 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:37:49.168336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:37:49.169412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:37:49.170357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:37:49.175470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 16/29] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:37:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:37:49.178406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e469b31-31ff-431b-97af-8415bd609473 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:49.183565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16692a85-d1d3-49e7-b6ed-0b77ec8fd464 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:49.249112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d32941-b61b-405d-9d47-85ccd33b45d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:49.249752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d32941-b61b-405d-9d47-85ccd33b45d6 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:49.250028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d32941-b61b-405d-9d47-85ccd33b45d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:49.250469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d32941-b61b-405d-9d47-85ccd33b45d6 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:49.251282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d32941-b61b-405d-9d47-85ccd33b45d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:37:49.251720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d32941-b61b-405d-9d47-85ccd33b45d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:49.253230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d32941-b61b-405d-9d47-85ccd33b45d6 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:49.265854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 14/30] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:37:49.266936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d32941-b61b-405d-9d47-85ccd33b45d6 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:49.332615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e469b31-31ff-431b-97af-8415bd609473 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:49.367763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:49.379948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b07db-d5f1-40d9-b255-7684736dcf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:49.380794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b07db-d5f1-40d9-b255-7684736dcf5e None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:49.381213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b07db-d5f1-40d9-b255-7684736dcf5e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:49.381652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b07db-d5f1-40d9-b255-7684736dcf5e None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:49.381786 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d32941-b61b-405d-9d47-85ccd33b45d6 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:49.382600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b07db-d5f1-40d9-b255-7684736dcf5e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:49.385161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b07db-d5f1-40d9-b255-7684736dcf5e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:49.389713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:49.422231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef07f0-3b05-46cf-b181-bff285874018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:49.423189 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef07f0-3b05-46cf-b181-bff285874018 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:49.423675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef07f0-3b05-46cf-b181-bff285874018 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:49.424225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef07f0-3b05-46cf-b181-bff285874018 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:49.425087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef07f0-3b05-46cf-b181-bff285874018 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:37:49.425648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef07f0-3b05-46cf-b181-bff285874018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:49.427194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef07f0-3b05-46cf-b181-bff285874018 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:49.440412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 15/31] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:37:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:37:49.442112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3a8c4-f76c-4c80-90d3-fee32067a62f None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:49.442592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3a8c4-f76c-4c80-90d3-fee32067a62f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:49.443054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3a8c4-f76c-4c80-90d3-fee32067a62f None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:49.444657 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 16/32] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:49.449559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3a8c4-f76c-4c80-90d3-fee32067a62f None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:49.529654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 17/33] 10.222.0.46 () {60 vars in 1131 bytes} [Mon Oct 3 14:37:49 2022] POST /identity/v3/roles => generated 219 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:37:49.531314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b07db-d5f1-40d9-b255-7684736dcf5e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:49.573929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3a8c4-f76c-4c80-90d3-fee32067a62f None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:49.615438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cca9c3-5bcc-4a6d-acb5-3b2a797c535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:49.616068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cca9c3-5bcc-4a6d-acb5-3b2a797c535b None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:49.616411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cca9c3-5bcc-4a6d-acb5-3b2a797c535b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:49.616686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cca9c3-5bcc-4a6d-acb5-3b2a797c535b None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:49.617931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cca9c3-5bcc-4a6d-acb5-3b2a797c535b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:49.625522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cca9c3-5bcc-4a6d-acb5-3b2a797c535b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:49.780418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b07db-d5f1-40d9-b255-7684736dcf5e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:49.786776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 17/34] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:37:49 2022] POST /identity/v3/projects => generated 301 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:37:49.788406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568aede-b122-4dd7-a16e-820ff8302725 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:49.788703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568aede-b122-4dd7-a16e-820ff8302725 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:49.788981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568aede-b122-4dd7-a16e-820ff8302725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:49.797969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:49.816878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-0568aede-b122-4dd7-a16e-820ff8302725 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:49.816878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0568aede-b122-4dd7-a16e-820ff8302725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:49.833220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d47a58-0f0c-49ac-98b6-5da42b18a089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:49.834129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d47a58-0f0c-49ac-98b6-5da42b18a089 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:49.834546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d47a58-0f0c-49ac-98b6-5da42b18a089 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:49.834979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d47a58-0f0c-49ac-98b6-5da42b18a089 None admin] PATH_INFO: `/v3/users/dd9397aad94548f284667a53c0b499d2` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:49.836409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d47a58-0f0c-49ac-98b6-5da42b18a089 None admin] RBAC: Authorizing `identity:get_user(user_id=dd9397aad94548f284667a53c0b499d2)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:49.838504 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d47a58-0f0c-49ac-98b6-5da42b18a089 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:49.839709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 18/35] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:37:49 2022] GET /identity/v3/users/dd9397aad94548f284667a53c0b499d2 => generated 247 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:37:49.846739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d47a58-0f0c-49ac-98b6-5da42b18a089 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:49.902372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 18/36] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:37:49.905462 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad999b-e24e-4642-80f6-10902b5d9b83 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:49.905775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad999b-e24e-4642-80f6-10902b5d9b83 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:49.906048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad999b-e24e-4642-80f6-10902b5d9b83 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:49.907625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 19/37] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:49.913215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9efb-4270-4a20-b782-d27ae36df7fb None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:49.913512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9efb-4270-4a20-b782-d27ae36df7fb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:49.913800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9efb-4270-4a20-b782-d27ae36df7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:49.931051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:49.934917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-84fa9efb-4270-4a20-b782-d27ae36df7fb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:49.936126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-84fa9efb-4270-4a20-b782-d27ae36df7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:50.003507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d47a58-0f0c-49ac-98b6-5da42b18a089 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:50.026172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 20/38] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:37:50.045994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90decea-c429-4c03-b6b0-30ef334abdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:50.046898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90decea-c429-4c03-b6b0-30ef334abdbc None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:50.047353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90decea-c429-4c03-b6b0-30ef334abdbc None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:50.047819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90decea-c429-4c03-b6b0-30ef334abdbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:50.049202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90decea-c429-4c03-b6b0-30ef334abdbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:50.050852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90decea-c429-4c03-b6b0-30ef334abdbc None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:50.052619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 19/39] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:37:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:37:50.058648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa9efb-4270-4a20-b782-d27ae36df7fb None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:50.101174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fab7dd-75e0-4e0c-b1b3-161e7340027f None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:50.101413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fab7dd-75e0-4e0c-b1b3-161e7340027f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:50.101977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fab7dd-75e0-4e0c-b1b3-161e7340027f None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:50.103349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 20/40] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:50.108766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1221-41e8-4d51-8f96-230ab0df9bcd None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:50.109122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1221-41e8-4d51-8f96-230ab0df9bcd None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:50.109929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1221-41e8-4d51-8f96-230ab0df9bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:50.131434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ecee1221-41e8-4d51-8f96-230ab0df9bcd None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:50.132933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ecee1221-41e8-4d51-8f96-230ab0df9bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:50.145803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:50.198397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa9efb-4270-4a20-b782-d27ae36df7fb None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:50.216269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 21/41] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:37:50.241688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fdca6-ab06-4de7-b11d-ca97c235c6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:50.242364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fdca6-ab06-4de7-b11d-ca97c235c6a5 None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:50.242922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fdca6-ab06-4de7-b11d-ca97c235c6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:50.243199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fdca6-ab06-4de7-b11d-ca97c235c6a5 None admin] PATH_INFO: `/v3/domains/default/users/dd9397aad94548f284667a53c0b499d2/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:50.248297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fdca6-ab06-4de7-b11d-ca97c235c6a5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=dd9397aad94548f284667a53c0b499d2, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:50.249924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fdca6-ab06-4de7-b11d-ca97c235c6a5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:50.269140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee1221-41e8-4d51-8f96-230ab0df9bcd None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:50.270061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 21/42] 10.222.0.46 () {58 vars in 1361 bytes} [Mon Oct 3 14:37:50 2022] PUT /identity/v3/domains/default/users/dd9397aad94548f284667a53c0b499d2/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:37:50.353664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fdca6-ab06-4de7-b11d-ca97c235c6a5 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:50.358356 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:50.411654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee1221-41e8-4d51-8f96-230ab0df9bcd None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:50.488216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fdca6-ab06-4de7-b11d-ca97c235c6a5 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:50.515356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:50.529043 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705786e-e4fd-4146-9031-4b1205c358b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:50.529750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705786e-e4fd-4146-9031-4b1205c358b0 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:50.530040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705786e-e4fd-4146-9031-4b1205c358b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:50.530318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705786e-e4fd-4146-9031-4b1205c358b0 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:50.531009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705786e-e4fd-4146-9031-4b1205c358b0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:50.532413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705786e-e4fd-4146-9031-4b1205c358b0 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:50.548408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 22/43] 10.222.0.46 () {60 vars in 1131 bytes} [Mon Oct 3 14:37:50 2022] POST /identity/v3/roles => generated 223 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:37:50.549228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705786e-e4fd-4146-9031-4b1205c358b0 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:50.566919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b288b-c106-4122-bb95-12cd5a5fa405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:50.567605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b288b-c106-4122-bb95-12cd5a5fa405 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:50.567953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b288b-c106-4122-bb95-12cd5a5fa405 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:50.568221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b288b-c106-4122-bb95-12cd5a5fa405 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:50.568973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b288b-c106-4122-bb95-12cd5a5fa405 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:50.571108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b288b-c106-4122-bb95-12cd5a5fa405 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:50.590966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 22/44] 10.222.0.46 () {60 vars in 1131 bytes} [Mon Oct 3 14:37:50 2022] POST /identity/v3/roles => generated 225 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:37:50.693264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705786e-e4fd-4146-9031-4b1205c358b0 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:50.735768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c4e37-6c6a-442a-b403-0840f61e2aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:50.736850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c4e37-6c6a-442a-b403-0840f61e2aed None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:50.737337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c4e37-6c6a-442a-b403-0840f61e2aed None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:50.737901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c4e37-6c6a-442a-b403-0840f61e2aed None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:50.739559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c4e37-6c6a-442a-b403-0840f61e2aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:50.741344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c4e37-6c6a-442a-b403-0840f61e2aed None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:50.743720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 23/45] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:37:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:37:50.750674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b288b-c106-4122-bb95-12cd5a5fa405 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:50.865246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b288b-c106-4122-bb95-12cd5a5fa405 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:50.887565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:50.913104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff5bb5-5ea3-4baa-98e4-6b0c6fd6fdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:50.914172 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff5bb5-5ea3-4baa-98e4-6b0c6fd6fdaa None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:50.914458 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff5bb5-5ea3-4baa-98e4-6b0c6fd6fdaa None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:50.914732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff5bb5-5ea3-4baa-98e4-6b0c6fd6fdaa None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:50.915393 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff5bb5-5ea3-4baa-98e4-6b0c6fd6fdaa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:50.916853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff5bb5-5ea3-4baa-98e4-6b0c6fd6fdaa None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:50.947472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 23/46] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:37:50 2022] POST /identity/v3/projects => generated 312 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:37:52.480550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450486a-b47d-4d6c-8bcb-128c24308621 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:52.481081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450486a-b47d-4d6c-8bcb-128c24308621 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:52.481161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450486a-b47d-4d6c-8bcb-128c24308621 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:52.483158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 24/47] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:52.488193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1830f1-5cb7-4de7-8daa-31b62eb50e7a None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:52.488562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1830f1-5cb7-4de7-8daa-31b62eb50e7a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:52.488875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1830f1-5cb7-4de7-8daa-31b62eb50e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:52.507492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-be1830f1-5cb7-4de7-8daa-31b62eb50e7a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:52.508667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-be1830f1-5cb7-4de7-8daa-31b62eb50e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:52.574086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 24/48] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:37:52.838705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450486a-b47d-4d6c-8bcb-128c24308621 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:52.889308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:52.940237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450486a-b47d-4d6c-8bcb-128c24308621 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:52.989864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f575a-1845-459e-92b3-1c146cc81bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:52.990521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f575a-1845-459e-92b3-1c146cc81bae None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:52.990809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f575a-1845-459e-92b3-1c146cc81bae None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:52.991080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f575a-1845-459e-92b3-1c146cc81bae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:53.083180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce592e-e408-42d8-8446-dceac68c78ef None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:53.083462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce592e-e408-42d8-8446-dceac68c78ef None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:53.083736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce592e-e408-42d8-8446-dceac68c78ef None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:53.085102 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 25/49] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:53.090913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78847743-11e4-4588-9d8b-645a19eb3255 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:53.091101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78847743-11e4-4588-9d8b-645a19eb3255 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:53.091347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78847743-11e4-4588-9d8b-645a19eb3255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:53.108804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f575a-1845-459e-92b3-1c146cc81bae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:37:53.110205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-78847743-11e4-4588-9d8b-645a19eb3255 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:53.111410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-78847743-11e4-4588-9d8b-645a19eb3255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-b46f575a-1845-459e-92b3-1c146cc81bae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:37:53.114836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:37:53.115619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:37:53.116362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 25/50] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:37:53.122044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f575a-1845-459e-92b3-1c146cc81bae None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:53.177385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 26/51] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:37:53.219022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f575a-1845-459e-92b3-1c146cc81bae None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:53.228704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:53.254347 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0b042-97b2-4b67-87fb-a3b1cd5c3d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:53.254947 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0b042-97b2-4b67-87fb-a3b1cd5c3d5d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:53.255227 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0b042-97b2-4b67-87fb-a3b1cd5c3d5d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:53.255498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0b042-97b2-4b67-87fb-a3b1cd5c3d5d None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:53.256162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0b042-97b2-4b67-87fb-a3b1cd5c3d5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:37:53.256463 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0b042-97b2-4b67-87fb-a3b1cd5c3d5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:53.257872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0b042-97b2-4b67-87fb-a3b1cd5c3d5d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:53.266015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 26/52] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:37:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:37:53.271688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78847743-11e4-4588-9d8b-645a19eb3255 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:53.302727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:53.373941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78847743-11e4-4588-9d8b-645a19eb3255 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:53.405974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cb8e9-3682-41dc-9f07-5a3f2429189f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:53.406605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cb8e9-3682-41dc-9f07-5a3f2429189f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:53.406885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cb8e9-3682-41dc-9f07-5a3f2429189f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:53.407157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cb8e9-3682-41dc-9f07-5a3f2429189f None admin] PATH_INFO: `/v3/users/dd9397aad94548f284667a53c0b499d2` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:53.408270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cb8e9-3682-41dc-9f07-5a3f2429189f None admin] RBAC: Authorizing `identity:get_user(user_id=dd9397aad94548f284667a53c0b499d2)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:53.409738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cb8e9-3682-41dc-9f07-5a3f2429189f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:53.411300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 27/53] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:37:53 2022] GET /identity/v3/users/dd9397aad94548f284667a53c0b499d2 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:37:53.416729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0b042-97b2-4b67-87fb-a3b1cd5c3d5d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:53.457214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cb8e9-3682-41dc-9f07-5a3f2429189f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:53.514141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0b042-97b2-4b67-87fb-a3b1cd5c3d5d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:53.548097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead5ddc-8f2d-4ac8-932a-7c5beba53d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:53.548815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead5ddc-8f2d-4ac8-932a-7c5beba53d55 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:53.549099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead5ddc-8f2d-4ac8-932a-7c5beba53d55 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:53.549370 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead5ddc-8f2d-4ac8-932a-7c5beba53d55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:53.550498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead5ddc-8f2d-4ac8-932a-7c5beba53d55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:53.551949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead5ddc-8f2d-4ac8-932a-7c5beba53d55 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:53.553376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:53.553803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 27/54] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:37:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:37:53.556592 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cb8e9-3682-41dc-9f07-5a3f2429189f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:53.559221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead5ddc-8f2d-4ac8-932a-7c5beba53d55 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:53.590423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b17f73-8faa-4f5b-8bf5-18737fcfbc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:53.591532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b17f73-8faa-4f5b-8bf5-18737fcfbc53 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:53.591821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b17f73-8faa-4f5b-8bf5-18737fcfbc53 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:53.592076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b17f73-8faa-4f5b-8bf5-18737fcfbc53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:53.593174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b17f73-8faa-4f5b-8bf5-18737fcfbc53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:53.594627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b17f73-8faa-4f5b-8bf5-18737fcfbc53 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:53.596178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 28/55] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:37:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:37:53.599314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:53.601955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b17f73-8faa-4f5b-8bf5-18737fcfbc53 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:53.658525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead5ddc-8f2d-4ac8-932a-7c5beba53d55 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:53.692700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e84f31-c4ea-4df7-8e16-2d12a6b570eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:53.693311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e84f31-c4ea-4df7-8e16-2d12a6b570eb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:53.693632 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e84f31-c4ea-4df7-8e16-2d12a6b570eb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:53.693920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e84f31-c4ea-4df7-8e16-2d12a6b570eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:53.694671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e84f31-c4ea-4df7-8e16-2d12a6b570eb None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.domain.id=default, user.id=dd9397aad94548f284667a53c0b499d2) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:37:53.694986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e84f31-c4ea-4df7-8e16-2d12a6b570eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:53.696438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e84f31-c4ea-4df7-8e16-2d12a6b570eb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:53.703910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b17f73-8faa-4f5b-8bf5-18737fcfbc53 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:53.704762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 28/56] 10.222.0.46 () {56 vars in 1322 bytes} [Mon Oct 3 14:37:53 2022] GET /identity/v3/role_assignments?user.id=dd9397aad94548f284667a53c0b499d2&scope.domain.id=default&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:37:53.739130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330dd787-928e-4ae4-ad26-460b8bf42471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:53.739751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dd787-928e-4ae4-ad26-460b8bf42471 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:53.740047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dd787-928e-4ae4-ad26-460b8bf42471 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:53.740304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dd787-928e-4ae4-ad26-460b8bf42471 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:53.740975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330dd787-928e-4ae4-ad26-460b8bf42471 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:53.742737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330dd787-928e-4ae4-ad26-460b8bf42471 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:53.771186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 29/57] 10.222.0.46 () {60 vars in 1140 bytes} [Mon Oct 3 14:37:53 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:37:55.912934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f84a9-d740-452d-9af9-1108b3b9a236 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:55.913446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f84a9-d740-452d-9af9-1108b3b9a236 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:55.913559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f84a9-d740-452d-9af9-1108b3b9a236 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:55.914838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 29/58] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:55.920491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb48ea-757f-45c5-8499-408cfa3d4be3 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:55.920919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb48ea-757f-45c5-8499-408cfa3d4be3 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:55.921042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb48ea-757f-45c5-8499-408cfa3d4be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:55.938776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-aafb48ea-757f-45c5-8499-408cfa3d4be3 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:37:55.939950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-aafb48ea-757f-45c5-8499-408cfa3d4be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:37:55.954322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:56.006909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 30/59] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:37:56.273076 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f84a9-d740-452d-9af9-1108b3b9a236 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:56.372893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f84a9-d740-452d-9af9-1108b3b9a236 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:56.375191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:37:56.405889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da024376-32bc-4391-813d-3eeef70362f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:56.406517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da024376-32bc-4391-813d-3eeef70362f3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:56.406801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da024376-32bc-4391-813d-3eeef70362f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:56.407076 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da024376-32bc-4391-813d-3eeef70362f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:56.408182 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da024376-32bc-4391-813d-3eeef70362f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:56.409685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da024376-32bc-4391-813d-3eeef70362f3 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:56.411259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 30/60] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:37:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:37:56.416780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb48ea-757f-45c5-8499-408cfa3d4be3 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:37:56.511558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb48ea-757f-45c5-8499-408cfa3d4be3 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:37:56.544412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4b6e3-e74f-4a3e-8439-78b09f3f7c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:37:56.544968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4b6e3-e74f-4a3e-8439-78b09f3f7c52 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:56.545248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4b6e3-e74f-4a3e-8439-78b09f3f7c52 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:56.545540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4b6e3-e74f-4a3e-8439-78b09f3f7c52 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:56.546185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4b6e3-e74f-4a3e-8439-78b09f3f7c52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:37:56.547634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4b6e3-e74f-4a3e-8439-78b09f3f7c52 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:37:56.573115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 31/61] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:37:56 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:37:59.987192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e8820-5513-4069-9459-6da6f086a2ee None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:59.988231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e8820-5513-4069-9459-6da6f086a2ee None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:59.988900 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e8820-5513-4069-9459-6da6f086a2ee None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:59.990580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 31/62] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:37:59.993120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ca830-1c2d-43ce-ac3d-ede44f2efabb None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:37:59.993960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ca830-1c2d-43ce-ac3d-ede44f2efabb None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:37:59.994431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ca830-1c2d-43ce-ac3d-ede44f2efabb None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:37:59.996175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 32/63] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:00.001439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e1499-49ec-451c-b123-db7008aba7b3 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.001977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e1499-49ec-451c-b123-db7008aba7b3 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.002457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e1499-49ec-451c-b123-db7008aba7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.024391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be98abe-9aa6-4f22-9675-17e670dbdfea None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.024988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be98abe-9aa6-4f22-9675-17e670dbdfea None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.025520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be98abe-9aa6-4f22-9675-17e670dbdfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.031360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-5f4e1499-49ec-451c-b123-db7008aba7b3 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:00.034288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5f4e1499-49ec-451c-b123-db7008aba7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:00.043005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:00.058979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-2be98abe-9aa6-4f22-9675-17e670dbdfea None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:00.062511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2be98abe-9aa6-4f22-9675-17e670dbdfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:00.169734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 32/64] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:00.195257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b2dde-9d3c-4748-bc13-0294816f43f7 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.195561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b2dde-9d3c-4748-bc13-0294816f43f7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.195829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b2dde-9d3c-4748-bc13-0294816f43f7 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.197420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 33/65] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:00.198981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 33/66] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:00.223309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d548a-678b-4800-b8ed-e510d726c324 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.223601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d548a-678b-4800-b8ed-e510d726c324 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.223887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d548a-678b-4800-b8ed-e510d726c324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.242838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-048d548a-678b-4800-b8ed-e510d726c324 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:00.243963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-048d548a-678b-4800-b8ed-e510d726c324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:00.279714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c50b00-c288-41d8-8fc2-acafe9b7bf06 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.280324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c50b00-c288-41d8-8fc2-acafe9b7bf06 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.280799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c50b00-c288-41d8-8fc2-acafe9b7bf06 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.282401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 34/67] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:38:00.290386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf2352-d8de-49fb-9323-67fd2deb0410 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.290668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf2352-d8de-49fb-9323-67fd2deb0410 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.290945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf2352-d8de-49fb-9323-67fd2deb0410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.323953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-43cf2352-d8de-49fb-9323-67fd2deb0410 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:00.325218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-43cf2352-d8de-49fb-9323-67fd2deb0410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:00.354763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 34/68] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:00.439822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 35/69] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:00.478244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d548a-678b-4800-b8ed-e510d726c324 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:00.500994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9c27-c00c-409c-aa85-7f909cf19506 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.501900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9c27-c00c-409c-aa85-7f909cf19506 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.502374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9c27-c00c-409c-aa85-7f909cf19506 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.504159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 36/70] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:38:00.508949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b84809-824f-4d6d-8928-d5d1205323ac None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.509916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b84809-824f-4d6d-8928-d5d1205323ac None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.510390 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b84809-824f-4d6d-8928-d5d1205323ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.525837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:00.547148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-a2b84809-824f-4d6d-8928-d5d1205323ac None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:00.548345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a2b84809-824f-4d6d-8928-d5d1205323ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:00.638651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d548a-678b-4800-b8ed-e510d726c324 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:00.649060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:00.657419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 37/71] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:00.658457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b84809-824f-4d6d-8928-d5d1205323ac None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:00.682629 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d4aec-82c1-4a00-90df-b2c778e0a50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:00.683802 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d4aec-82c1-4a00-90df-b2c778e0a50f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.684288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d4aec-82c1-4a00-90df-b2c778e0a50f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.684737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d4aec-82c1-4a00-90df-b2c778e0a50f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.698240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d4aec-82c1-4a00-90df-b2c778e0a50f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-6a1d4aec-82c1-4a00-90df-b2c778e0a50f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:00.704685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:00.705545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:00.706327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:00.710825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 35/72] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:00.711945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d4aec-82c1-4a00-90df-b2c778e0a50f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:00.776813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b84809-824f-4d6d-8928-d5d1205323ac None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:00.811800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90df30-431e-423f-a4b6-c3c56716e1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:00.812433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90df30-431e-423f-a4b6-c3c56716e1e9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.812717 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90df30-431e-423f-a4b6-c3c56716e1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.812991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90df30-431e-423f-a4b6-c3c56716e1e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.823320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90df30-431e-423f-a4b6-c3c56716e1e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-5c90df30-431e-423f-a4b6-c3c56716e1e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:00.828208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:00.829031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:00.830576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:00.831430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:00.831510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 38/73] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:00.831510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90df30-431e-423f-a4b6-c3c56716e1e9 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:00.843787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d4aec-82c1-4a00-90df-b2c778e0a50f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:00.882462 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3b4bb-3830-4e5f-9862-cff857de4863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:00.883105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3b4bb-3830-4e5f-9862-cff857de4863 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.883392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3b4bb-3830-4e5f-9862-cff857de4863 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.883668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3b4bb-3830-4e5f-9862-cff857de4863 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.893163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:00.896144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3b4bb-3830-4e5f-9862-cff857de4863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-85b3b4bb-3830-4e5f-9862-cff857de4863 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:00.901949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:00.903325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:00.904084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 36/74] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:38:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:00.905027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3b4bb-3830-4e5f-9862-cff857de4863 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:00.943656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90df30-431e-423f-a4b6-c3c56716e1e9 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:00.976970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53abef-33c2-4a9c-8e11-6e745b63d94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:00.977594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53abef-33c2-4a9c-8e11-6e745b63d94e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:00.977906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53abef-33c2-4a9c-8e11-6e745b63d94e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:00.978176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53abef-33c2-4a9c-8e11-6e745b63d94e None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:00.978838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53abef-33c2-4a9c-8e11-6e745b63d94e None admin] RBAC: Adding query filter params (name=member) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:00.979137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53abef-33c2-4a9c-8e11-6e745b63d94e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:00.980533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53abef-33c2-4a9c-8e11-6e745b63d94e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:00.988964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 39/75] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:00.989771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53abef-33c2-4a9c-8e11-6e745b63d94e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.006364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3b4bb-3830-4e5f-9862-cff857de4863 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.039025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298593a-d827-450e-9471-dae7ff825cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.039590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298593a-d827-450e-9471-dae7ff825cb4 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.039876 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298593a-d827-450e-9471-dae7ff825cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.040157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298593a-d827-450e-9471-dae7ff825cb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.050161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298593a-d827-450e-9471-dae7ff825cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-6298593a-d827-450e-9471-dae7ff825cb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:01.055052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:01.056051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:01.056797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 37/76] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:01.057425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298593a-d827-450e-9471-dae7ff825cb4 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.072831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:01.088297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53abef-33c2-4a9c-8e11-6e745b63d94e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.108807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:01.124405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5b876-5457-4f5a-8432-343d7b0d298a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.125013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5b876-5457-4f5a-8432-343d7b0d298a None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.125289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5b876-5457-4f5a-8432-343d7b0d298a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.125589 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5b876-5457-4f5a-8432-343d7b0d298a None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.126248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5b876-5457-4f5a-8432-343d7b0d298a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:01.126552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5b876-5457-4f5a-8432-343d7b0d298a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:01.127933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5b876-5457-4f5a-8432-343d7b0d298a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:01.135759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 40/77] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:01.136696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5b876-5457-4f5a-8432-343d7b0d298a None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.163224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298593a-d827-450e-9471-dae7ff825cb4 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.196330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198ae71-5719-4e32-9615-2f82829182b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.196942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198ae71-5719-4e32-9615-2f82829182b5 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.197225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198ae71-5719-4e32-9615-2f82829182b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.197524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198ae71-5719-4e32-9615-2f82829182b5 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.198199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198ae71-5719-4e32-9615-2f82829182b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:01.198508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198ae71-5719-4e32-9615-2f82829182b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:01.200421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198ae71-5719-4e32-9615-2f82829182b5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:01.208211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 38/78] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:01.213023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198ae71-5719-4e32-9615-2f82829182b5 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.233939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5b876-5457-4f5a-8432-343d7b0d298a None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.266564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd06da5-6200-4e2b-94b4-fed9a1cec487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.267195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd06da5-6200-4e2b-94b4-fed9a1cec487 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.267473 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd06da5-6200-4e2b-94b4-fed9a1cec487 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.267749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd06da5-6200-4e2b-94b4-fed9a1cec487 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.268857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd06da5-6200-4e2b-94b4-fed9a1cec487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:01.270341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd06da5-6200-4e2b-94b4-fed9a1cec487 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:01.271926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 41/79] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:01.272682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd06da5-6200-4e2b-94b4-fed9a1cec487 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.286357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:01.317107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198ae71-5719-4e32-9615-2f82829182b5 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.325647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:01.350613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cdd90-7818-4ead-8962-7f5e11649c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.351249 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cdd90-7818-4ead-8962-7f5e11649c50 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.351533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cdd90-7818-4ead-8962-7f5e11649c50 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.351814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cdd90-7818-4ead-8962-7f5e11649c50 None admin] PATH_INFO: `/v3/users/0e508b7987364e59b4008c6b649a48e0` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.366169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cdd90-7818-4ead-8962-7f5e11649c50 None admin] RBAC: Authorizing `identity:get_user(user_id=0e508b7987364e59b4008c6b649a48e0)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:01.367912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cdd90-7818-4ead-8962-7f5e11649c50 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:01.369760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 39/80] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:01 2022] GET /identity/v3/users/0e508b7987364e59b4008c6b649a48e0 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:01.370312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cdd90-7818-4ead-8962-7f5e11649c50 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.376330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd06da5-6200-4e2b-94b4-fed9a1cec487 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.409875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce5d34-b142-4807-86b6-22c349c7f0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.410478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce5d34-b142-4807-86b6-22c349c7f0a6 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.410756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce5d34-b142-4807-86b6-22c349c7f0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.411024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce5d34-b142-4807-86b6-22c349c7f0a6 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.411685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce5d34-b142-4807-86b6-22c349c7f0a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:01.411983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce5d34-b142-4807-86b6-22c349c7f0a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:01.413349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce5d34-b142-4807-86b6-22c349c7f0a6 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:01.421463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 42/81] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:01.422259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce5d34-b142-4807-86b6-22c349c7f0a6 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.467671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cdd90-7818-4ead-8962-7f5e11649c50 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.498361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:01.504277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2aa1e9-76c5-4c9a-8a01-c6de9645cdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.504907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2aa1e9-76c5-4c9a-8a01-c6de9645cdbd None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.505186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2aa1e9-76c5-4c9a-8a01-c6de9645cdbd None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.505466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2aa1e9-76c5-4c9a-8a01-c6de9645cdbd None admin] PATH_INFO: `/v3/users/0e508b7987364e59b4008c6b649a48e0` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.506606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2aa1e9-76c5-4c9a-8a01-c6de9645cdbd None admin] RBAC: Authorizing `identity:get_user(user_id=0e508b7987364e59b4008c6b649a48e0)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:01.508058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2aa1e9-76c5-4c9a-8a01-c6de9645cdbd None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:01.509646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 40/82] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:01 2022] GET /identity/v3/users/0e508b7987364e59b4008c6b649a48e0 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:01.510380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2aa1e9-76c5-4c9a-8a01-c6de9645cdbd None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.522989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce5d34-b142-4807-86b6-22c349c7f0a6 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.532863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:01.556148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d410e3-1de0-4d43-a2ff-ebf05fe47b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.556708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d410e3-1de0-4d43-a2ff-ebf05fe47b50 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.556982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d410e3-1de0-4d43-a2ff-ebf05fe47b50 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.557259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d410e3-1de0-4d43-a2ff-ebf05fe47b50 None admin] PATH_INFO: `/v3/users/dd9397aad94548f284667a53c0b499d2` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.558385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d410e3-1de0-4d43-a2ff-ebf05fe47b50 None admin] RBAC: Authorizing `identity:get_user(user_id=dd9397aad94548f284667a53c0b499d2)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:01.560403 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d410e3-1de0-4d43-a2ff-ebf05fe47b50 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:01.562141 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 43/83] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:01 2022] GET /identity/v3/users/dd9397aad94548f284667a53c0b499d2 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:01.562733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d410e3-1de0-4d43-a2ff-ebf05fe47b50 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.614160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2aa1e9-76c5-4c9a-8a01-c6de9645cdbd None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.646753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a6220-52c7-48ac-917d-818810309f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.647359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a6220-52c7-48ac-917d-818810309f75 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.647640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a6220-52c7-48ac-917d-818810309f75 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.647915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a6220-52c7-48ac-917d-818810309f75 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.648565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a6220-52c7-48ac-917d-818810309f75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:01.650054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a6220-52c7-48ac-917d-818810309f75 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:01.657413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d410e3-1de0-4d43-a2ff-ebf05fe47b50 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.691646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46adb8-b53e-445b-ab3d-ce18c587a53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.692265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46adb8-b53e-445b-ab3d-ce18c587a53c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.692553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46adb8-b53e-445b-ab3d-ce18c587a53c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.692752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 41/84] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:38:01 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:38:01.692836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46adb8-b53e-445b-ab3d-ce18c587a53c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.693609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a6220-52c7-48ac-917d-818810309f75 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.753007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:01.762603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:01.782747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46adb8-b53e-445b-ab3d-ce18c587a53c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:01.784309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46adb8-b53e-445b-ab3d-ce18c587a53c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:01.815512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a6220-52c7-48ac-917d-818810309f75 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:01.853145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee298b-6438-4ca4-b3e4-93218193f22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:01.853825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee298b-6438-4ca4-b3e4-93218193f22a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:01.854106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee298b-6438-4ca4-b3e4-93218193f22a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:01.854377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee298b-6438-4ca4-b3e4-93218193f22a None admin] PATH_INFO: `/v3/users/0e508b7987364e59b4008c6b649a48e0` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:01.855636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee298b-6438-4ca4-b3e4-93218193f22a None admin] RBAC: Authorizing `identity:get_user(user_id=0e508b7987364e59b4008c6b649a48e0)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:01.857687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee298b-6438-4ca4-b3e4-93218193f22a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:01.863676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 42/85] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:01 2022] GET /identity/v3/users/0e508b7987364e59b4008c6b649a48e0 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:01.863676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee298b-6438-4ca4-b3e4-93218193f22a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.908279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 44/86] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:01.909366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46adb8-b53e-445b-ab3d-ce18c587a53c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:01.977777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee298b-6438-4ca4-b3e4-93218193f22a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:02.004551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:02.008412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:02.025591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391428f-8e3e-4d90-8b61-44b18dcdc0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:02.026241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391428f-8e3e-4d90-8b61-44b18dcdc0cb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:02.026518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391428f-8e3e-4d90-8b61-44b18dcdc0cb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:02.026798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391428f-8e3e-4d90-8b61-44b18dcdc0cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:02.035500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46adb8-b53e-445b-ab3d-ce18c587a53c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:02.072819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddc267-6d5a-4ace-b8bc-60d2e1698a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:02.073835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddc267-6d5a-4ace-b8bc-60d2e1698a3f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:02.074351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddc267-6d5a-4ace-b8bc-60d2e1698a3f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:02.075247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddc267-6d5a-4ace-b8bc-60d2e1698a3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:02.110871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391428f-8e3e-4d90-8b61-44b18dcdc0cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:02.112492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391428f-8e3e-4d90-8b61-44b18dcdc0cb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:02.161786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddc267-6d5a-4ace-b8bc-60d2e1698a3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:02.164248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddc267-6d5a-4ace-b8bc-60d2e1698a3f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:02.221143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 43/87] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:02.222335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391428f-8e3e-4d90-8b61-44b18dcdc0cb None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:02.251671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:02.279259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:02.317860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 45/88] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:02.323212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddc267-6d5a-4ace-b8bc-60d2e1698a3f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:02.345320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391428f-8e3e-4d90-8b61-44b18dcdc0cb None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:02.386317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388ae91-8d00-4017-9caf-af872a828839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:02.386944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388ae91-8d00-4017-9caf-af872a828839 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:02.387836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388ae91-8d00-4017-9caf-af872a828839 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:02.388397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388ae91-8d00-4017-9caf-af872a828839 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:02.452324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddc267-6d5a-4ace-b8bc-60d2e1698a3f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:02.468333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388ae91-8d00-4017-9caf-af872a828839 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:02.470118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388ae91-8d00-4017-9caf-af872a828839 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:02.494582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02376d-feab-45c3-95bb-2f1bb31e69b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:02.495252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02376d-feab-45c3-95bb-2f1bb31e69b3 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:02.496131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:02.496258 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02376d-feab-45c3-95bb-2f1bb31e69b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:02.497100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02376d-feab-45c3-95bb-2f1bb31e69b3 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:02.499841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02376d-feab-45c3-95bb-2f1bb31e69b3 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:02.502723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02376d-feab-45c3-95bb-2f1bb31e69b3 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:02.505375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 46/89] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:02 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:02.506189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02376d-feab-45c3-95bb-2f1bb31e69b3 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:02.569939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:02.582650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 44/90] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:02.583901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388ae91-8d00-4017-9caf-af872a828839 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:02.645664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02376d-feab-45c3-95bb-2f1bb31e69b3 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:02.696011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c00d78-2fa4-4972-98cf-7211e8b29cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:02.697789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c00d78-2fa4-4972-98cf-7211e8b29cc2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:02.698353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c00d78-2fa4-4972-98cf-7211e8b29cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:02.699063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c00d78-2fa4-4972-98cf-7211e8b29cc2 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:02.701653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c00d78-2fa4-4972-98cf-7211e8b29cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:02.704309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c00d78-2fa4-4972-98cf-7211e8b29cc2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:02.706020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388ae91-8d00-4017-9caf-af872a828839 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:02.706467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 47/91] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:02 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:02.707418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c00d78-2fa4-4972-98cf-7211e8b29cc2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:02.736296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:02.749915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138a1f-7c92-43f9-9a46-255a95886b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:02.750557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138a1f-7c92-43f9-9a46-255a95886b7d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:02.750844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138a1f-7c92-43f9-9a46-255a95886b7d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:02.751124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce138a1f-7c92-43f9-9a46-255a95886b7d None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:02.752378 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce138a1f-7c92-43f9-9a46-255a95886b7d None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:02.753922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce138a1f-7c92-43f9-9a46-255a95886b7d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:02.756135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 45/92] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:02 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:02.756996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138a1f-7c92-43f9-9a46-255a95886b7d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:02.830105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c00d78-2fa4-4972-98cf-7211e8b29cc2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:02.844256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:02.866428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfdaf6-d1c8-4d58-bb06-08c7bc0f8c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:02.867409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfdaf6-d1c8-4d58-bb06-08c7bc0f8c7b None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:02.867677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfdaf6-d1c8-4d58-bb06-08c7bc0f8c7b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:02.867952 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfdaf6-d1c8-4d58-bb06-08c7bc0f8c7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:02.868726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfdaf6-d1c8-4d58-bb06-08c7bc0f8c7b None admin] RBAC: Adding query filter params (role.id=47112770ffdc45638da090c45073888e, scope.project.id=130f64a6280e4b83bf885edf8986e895, user.id=0e508b7987364e59b4008c6b649a48e0) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:02.869056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfdaf6-d1c8-4d58-bb06-08c7bc0f8c7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:02.870531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfdaf6-d1c8-4d58-bb06-08c7bc0f8c7b None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:02.876854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 48/93] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:02 2022] GET /identity/v3/role_assignments?user.id=0e508b7987364e59b4008c6b649a48e0&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=47112770ffdc45638da090c45073888e => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:02.877522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfdaf6-d1c8-4d58-bb06-08c7bc0f8c7b None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:02.900575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce138a1f-7c92-43f9-9a46-255a95886b7d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:02.945064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce53af-6cb2-4a99-a8c7-455c12426f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:02.946819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce53af-6cb2-4a99-a8c7-455c12426f07 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:02.947322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce53af-6cb2-4a99-a8c7-455c12426f07 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:02.947794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce53af-6cb2-4a99-a8c7-455c12426f07 None admin] PATH_INFO: `/v3/projects/cd94cbfdffec47d597dde3e2bc2d86d5` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:02.949313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce53af-6cb2-4a99-a8c7-455c12426f07 None admin] RBAC: Authorizing `identity:get_project(project_id=cd94cbfdffec47d597dde3e2bc2d86d5)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:02.951581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce53af-6cb2-4a99-a8c7-455c12426f07 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:02.954263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 46/94] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:02 2022] GET /identity/v3/projects/cd94cbfdffec47d597dde3e2bc2d86d5 => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:02.955588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce53af-6cb2-4a99-a8c7-455c12426f07 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:03.032542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:03.076247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfdaf6-d1c8-4d58-bb06-08c7bc0f8c7b None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:03.093431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce53af-6cb2-4a99-a8c7-455c12426f07 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:03.115362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da260c9-5768-4539-8ead-dad4c76af9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:03.122274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da260c9-5768-4539-8ead-dad4c76af9c2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:03.122553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da260c9-5768-4539-8ead-dad4c76af9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:03.122838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da260c9-5768-4539-8ead-dad4c76af9c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:03.123597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da260c9-5768-4539-8ead-dad4c76af9c2 None admin] RBAC: Adding query filter params (role.id=8cd21d388193490bac3aa386bcc18763, scope.project.id=130f64a6280e4b83bf885edf8986e895, user.id=0e508b7987364e59b4008c6b649a48e0) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:03.123906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da260c9-5768-4539-8ead-dad4c76af9c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:03.125386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da260c9-5768-4539-8ead-dad4c76af9c2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:03.139114 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d586f5-9f62-494f-9495-6994386f5275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:03.140031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d586f5-9f62-494f-9495-6994386f5275 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:03.140380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 49/95] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:02 2022] GET /identity/v3/role_assignments?user.id=0e508b7987364e59b4008c6b649a48e0&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=8cd21d388193490bac3aa386bcc18763 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:03.140741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d586f5-9f62-494f-9495-6994386f5275 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:03.141209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d586f5-9f62-494f-9495-6994386f5275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:03.141308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da260c9-5768-4539-8ead-dad4c76af9c2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:03.142364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d586f5-9f62-494f-9495-6994386f5275 None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=130f64a6280e4b83bf885edf8986e895, user.id=dd9397aad94548f284667a53c0b499d2) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:03.142878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d586f5-9f62-494f-9495-6994386f5275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:03.144589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d586f5-9f62-494f-9495-6994386f5275 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:03.151611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 47/96] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:02 2022] GET /identity/v3/role_assignments?user.id=dd9397aad94548f284667a53c0b499d2&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:03.201728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:03.274911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da260c9-5768-4539-8ead-dad4c76af9c2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:03.317786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d57bd-2e08-481f-89d7-b65421d275ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:03.318413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d57bd-2e08-481f-89d7-b65421d275ce None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:03.318694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d57bd-2e08-481f-89d7-b65421d275ce None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:03.318963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d57bd-2e08-481f-89d7-b65421d275ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:03.319701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d57bd-2e08-481f-89d7-b65421d275ce None admin] RBAC: Adding query filter params (role.id=47112770ffdc45638da090c45073888e, scope.project.id=cd94cbfdffec47d597dde3e2bc2d86d5, user.id=0e508b7987364e59b4008c6b649a48e0) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:03.320005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d57bd-2e08-481f-89d7-b65421d275ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:03.321455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d57bd-2e08-481f-89d7-b65421d275ce None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:03.328034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 50/97] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:03 2022] GET /identity/v3/role_assignments?user.id=0e508b7987364e59b4008c6b649a48e0&scope.project.id=cd94cbfdffec47d597dde3e2bc2d86d5&role.id=47112770ffdc45638da090c45073888e => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:04.384951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219a4a-4817-4cb8-b1d0-8a952e4b9959 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:04.384951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219a4a-4817-4cb8-b1d0-8a952e4b9959 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:04.384951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219a4a-4817-4cb8-b1d0-8a952e4b9959 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:04.385987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 48/98] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:04.400861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4b1db-b365-4a8b-9e18-a1bcb9e96ffb None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:04.401318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4b1db-b365-4a8b-9e18-a1bcb9e96ffb None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:04.401560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4b1db-b365-4a8b-9e18-a1bcb9e96ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:04.430512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c0b4b1db-b365-4a8b-9e18-a1bcb9e96ffb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:04.431710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c0b4b1db-b365-4a8b-9e18-a1bcb9e96ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:04.531274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 51/99] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:04.618627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb7fd8-c39e-4c3a-b305-b74a0308e596 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:04.618627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb7fd8-c39e-4c3a-b305-b74a0308e596 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:04.618627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb7fd8-c39e-4c3a-b305-b74a0308e596 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:04.618627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 49/100] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:04.624368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bb5ea-7291-440d-b128-444009afce4b None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:04.624655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bb5ea-7291-440d-b128-444009afce4b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:04.624932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bb5ea-7291-440d-b128-444009afce4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:04.650457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-823bb5ea-7291-440d-b128-444009afce4b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:04.651644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-823bb5ea-7291-440d-b128-444009afce4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:04.685870 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:04.725293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 52/101] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:05.066354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb7fd8-c39e-4c3a-b305-b74a0308e596 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:05.089930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bb5ea-7291-440d-b128-444009afce4b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:05.206203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb7fd8-c39e-4c3a-b305-b74a0308e596 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:05.248686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bb5ea-7291-440d-b128-444009afce4b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:05.258891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:05.287255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4daa2-d7f2-4340-b4d3-5e91905d71e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:05.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4daa2-d7f2-4340-b4d3-5e91905d71e0 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:05.288256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4daa2-d7f2-4340-b4d3-5e91905d71e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:05.288529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4daa2-d7f2-4340-b4d3-5e91905d71e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:05.321132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764556-2c78-47e1-b1cc-876608628378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:05.321729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764556-2c78-47e1-b1cc-876608628378 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:05.322012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764556-2c78-47e1-b1cc-876608628378 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:05.322288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764556-2c78-47e1-b1cc-876608628378 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:05.322567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4daa2-d7f2-4340-b4d3-5e91905d71e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:05.323504 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93764556-2c78-47e1-b1cc-876608628378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:05.325063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93764556-2c78-47e1-b1cc-876608628378 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:05.327023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 53/102] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-9ed4daa2-d7f2-4340-b4d3-5e91905d71e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:05.331674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:05.332502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:05.333237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:05.333843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764556-2c78-47e1-b1cc-876608628378 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:05.333917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 50/103] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:05.354549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4daa2-d7f2-4340-b4d3-5e91905d71e0 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:05.388734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:05.522075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764556-2c78-47e1-b1cc-876608628378 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:05.576706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4daa2-d7f2-4340-b4d3-5e91905d71e0 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:05.595452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165d7b3-58e6-4ab4-b8c6-2dfda200e91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:05.596067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165d7b3-58e6-4ab4-b8c6-2dfda200e91e None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:05.596344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165d7b3-58e6-4ab4-b8c6-2dfda200e91e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:05.596616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165d7b3-58e6-4ab4-b8c6-2dfda200e91e None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:05.597278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165d7b3-58e6-4ab4-b8c6-2dfda200e91e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:05.599630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165d7b3-58e6-4ab4-b8c6-2dfda200e91e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:05.638586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335e122-b521-47d6-8dec-2da4e664db02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:05.639214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335e122-b521-47d6-8dec-2da4e664db02 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:05.639559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335e122-b521-47d6-8dec-2da4e664db02 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:05.639837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335e122-b521-47d6-8dec-2da4e664db02 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:05.640578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335e122-b521-47d6-8dec-2da4e664db02 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:05.646874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335e122-b521-47d6-8dec-2da4e664db02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:05.648285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335e122-b521-47d6-8dec-2da4e664db02 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:05.653277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 54/104] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:38:05 2022] POST /identity/v3/projects => generated 302 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:38:05.660077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 51/105] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:05.663482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460f11c-94e0-461d-83b4-99d47ba09157 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:05.663827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460f11c-94e0-461d-83b4-99d47ba09157 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:05.664106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460f11c-94e0-461d-83b4-99d47ba09157 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:05.665696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 55/106] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:05.666212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335e122-b521-47d6-8dec-2da4e664db02 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:05.671913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1829dd-bc58-4d8c-ab88-c0e49cd1c056 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:05.672197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1829dd-bc58-4d8c-ab88-c0e49cd1c056 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:05.672530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1829dd-bc58-4d8c-ab88-c0e49cd1c056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:05.692176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-df1829dd-bc58-4d8c-ab88-c0e49cd1c056 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:05.693529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-df1829dd-bc58-4d8c-ab88-c0e49cd1c056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:05.732985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:05.816726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 56/107] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:05.828366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335e122-b521-47d6-8dec-2da4e664db02 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:05.875622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf2270-eb90-4bfb-bfc1-cfd341055a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:05.876278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf2270-eb90-4bfb-bfc1-cfd341055a0d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:05.876556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf2270-eb90-4bfb-bfc1-cfd341055a0d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:05.876826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf2270-eb90-4bfb-bfc1-cfd341055a0d None admin] PATH_INFO: `/v3/users/780865e68bbc4cc49315105a3c0b90f7` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:05.901185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf2270-eb90-4bfb-bfc1-cfd341055a0d None admin] RBAC: Authorizing `identity:get_user(user_id=780865e68bbc4cc49315105a3c0b90f7)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:05.902854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf2270-eb90-4bfb-bfc1-cfd341055a0d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:05.904624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 52/108] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:05 2022] GET /identity/v3/users/780865e68bbc4cc49315105a3c0b90f7 => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:05.910510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1829dd-bc58-4d8c-ab88-c0e49cd1c056 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:05.935254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:06.027719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1829dd-bc58-4d8c-ab88-c0e49cd1c056 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:06.062176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc8982-9900-4e9d-87a4-6c6c6fdb1323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:06.062786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc8982-9900-4e9d-87a4-6c6c6fdb1323 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:06.063066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc8982-9900-4e9d-87a4-6c6c6fdb1323 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:06.063337 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc8982-9900-4e9d-87a4-6c6c6fdb1323 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:06.131197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc8982-9900-4e9d-87a4-6c6c6fdb1323 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:06.132587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc8982-9900-4e9d-87a4-6c6c6fdb1323 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:06.149796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:06.227073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf2270-eb90-4bfb-bfc1-cfd341055a0d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:06.265705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 57/109] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:06.273363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc8982-9900-4e9d-87a4-6c6c6fdb1323 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:06.441929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf2270-eb90-4bfb-bfc1-cfd341055a0d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:06.445706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:06.474627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc8982-9900-4e9d-87a4-6c6c6fdb1323 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:06.496473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3ad57-80fd-49d1-b6b1-07670fe97d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:06.497148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3ad57-80fd-49d1-b6b1-07670fe97d53 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:06.497432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3ad57-80fd-49d1-b6b1-07670fe97d53 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:06.497733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3ad57-80fd-49d1-b6b1-07670fe97d53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:06.509104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3ad57-80fd-49d1-b6b1-07670fe97d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:06.510580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-cfe3ad57-80fd-49d1-b6b1-07670fe97d53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:06.515612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:06.516605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:06.517540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:06.519606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 53/110] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:06.521264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689cff-cc93-4db2-9829-f622b1936fc9 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:06.521924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689cff-cc93-4db2-9829-f622b1936fc9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:06.522466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689cff-cc93-4db2-9829-f622b1936fc9 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:06.524830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 54/111] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:06.528887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e689cff-cc93-4db2-9829-f622b1936fc9 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:06.566288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc4e88-8618-4c37-8ac1-5d924a6196c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:06.566941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc4e88-8618-4c37-8ac1-5d924a6196c9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:06.567229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc4e88-8618-4c37-8ac1-5d924a6196c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:06.567505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc4e88-8618-4c37-8ac1-5d924a6196c9 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:06.572943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc4e88-8618-4c37-8ac1-5d924a6196c9 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:06.575034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc4e88-8618-4c37-8ac1-5d924a6196c9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:06.576913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 58/112] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:06 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:06.578264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1445df4-0175-44dd-91e4-ffaa8d6c4dfa None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:06.578694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1445df4-0175-44dd-91e4-ffaa8d6c4dfa None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:06.579112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1445df4-0175-44dd-91e4-ffaa8d6c4dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:06.611457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c1445df4-0175-44dd-91e4-ffaa8d6c4dfa None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:06.612869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c1445df4-0175-44dd-91e4-ffaa8d6c4dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:06.660069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e689cff-cc93-4db2-9829-f622b1936fc9 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:06.697803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5c45b-8136-436b-bc86-a8d703fe39d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:06.698342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5c45b-8136-436b-bc86-a8d703fe39d6 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:06.698625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5c45b-8136-436b-bc86-a8d703fe39d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:06.698897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5c45b-8136-436b-bc86-a8d703fe39d6 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:06.700106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5c45b-8136-436b-bc86-a8d703fe39d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:06.700412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5c45b-8136-436b-bc86-a8d703fe39d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:06.701875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5c45b-8136-436b-bc86-a8d703fe39d6 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:06.710870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 55/113] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:06.711703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5c45b-8136-436b-bc86-a8d703fe39d6 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:06.732989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 59/114] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:06.734266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93a09-72bd-44c1-aa7a-94ccdd84d239 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:06.734552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93a09-72bd-44c1-aa7a-94ccdd84d239 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:06.734822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93a09-72bd-44c1-aa7a-94ccdd84d239 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:06.736217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 60/115] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:06.740912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c93a09-72bd-44c1-aa7a-94ccdd84d239 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:06.746203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:06.822021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5c45b-8136-436b-bc86-a8d703fe39d6 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:06.855400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c93a09-72bd-44c1-aa7a-94ccdd84d239 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:06.857034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d05c9a-308e-4ca2-8337-fb940fa1bb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:06.857808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d05c9a-308e-4ca2-8337-fb940fa1bb93 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:06.858164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d05c9a-308e-4ca2-8337-fb940fa1bb93 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:06.858512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d05c9a-308e-4ca2-8337-fb940fa1bb93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:06.859373 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d05c9a-308e-4ca2-8337-fb940fa1bb93 None admin] RBAC: Adding query filter params (role.id=8d46737ab3ff453a800fe6b47805eb5a, scope.project.id=130f64a6280e4b83bf885edf8986e895, user.id=780865e68bbc4cc49315105a3c0b90f7) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:06.859769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d05c9a-308e-4ca2-8337-fb940fa1bb93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:06.861336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d05c9a-308e-4ca2-8337-fb940fa1bb93 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:06.867721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 56/116] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:06 2022] GET /identity/v3/role_assignments?user.id=780865e68bbc4cc49315105a3c0b90f7&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=8d46737ab3ff453a800fe6b47805eb5a => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:06.869161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51173e-de3c-420d-bf90-9ce1c0a70f7f None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:06.869547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51173e-de3c-420d-bf90-9ce1c0a70f7f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:06.869846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51173e-de3c-420d-bf90-9ce1c0a70f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:06.870809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:06.894855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0e18c-8646-47d2-845b-ebd584eba928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:06.895505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0e18c-8646-47d2-845b-ebd584eba928 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:06.895786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0e18c-8646-47d2-845b-ebd584eba928 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:06.896060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0e18c-8646-47d2-845b-ebd584eba928 None admin] PATH_INFO: `/v3/users/0e508b7987364e59b4008c6b649a48e0` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:06.897225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0e18c-8646-47d2-845b-ebd584eba928 None admin] RBAC: Authorizing `identity:get_user(user_id=0e508b7987364e59b4008c6b649a48e0)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:06.899251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0e18c-8646-47d2-845b-ebd584eba928 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:06.901913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 61/117] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:06 2022] GET /identity/v3/users/0e508b7987364e59b4008c6b649a48e0 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:06.908121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0e18c-8646-47d2-845b-ebd584eba928 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:06.908375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9e51173e-de3c-420d-bf90-9ce1c0a70f7f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:06.909702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9e51173e-de3c-420d-bf90-9ce1c0a70f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:07.021790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0e18c-8646-47d2-845b-ebd584eba928 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:07.026088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 57/118] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:07.081011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d1fdd-c0a1-4246-a91c-06febab6f537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:07.081788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d1fdd-c0a1-4246-a91c-06febab6f537 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:07.082140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d1fdd-c0a1-4246-a91c-06febab6f537 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:07.082483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d1fdd-c0a1-4246-a91c-06febab6f537 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:07.086207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d1fdd-c0a1-4246-a91c-06febab6f537 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:07.087768 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d1fdd-c0a1-4246-a91c-06febab6f537 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:07.089548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 62/119] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:06 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:07.096234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51173e-de3c-420d-bf90-9ce1c0a70f7f None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:07.126816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:07.211203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51173e-de3c-420d-bf90-9ce1c0a70f7f None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:07.247427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493b133-f77e-469b-8bac-049e0742acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:07.248101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493b133-f77e-469b-8bac-049e0742acb5 None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:07.248383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493b133-f77e-469b-8bac-049e0742acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:07.248658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493b133-f77e-469b-8bac-049e0742acb5 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895/users/0e508b7987364e59b4008c6b649a48e0/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:07.250708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493b133-f77e-469b-8bac-049e0742acb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=130f64a6280e4b83bf885edf8986e895, user_id=0e508b7987364e59b4008c6b649a48e0, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:07.252044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389006e5-def2-4e4e-8c57-b3a1a8ea7c4c None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:07.252286 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493b133-f77e-469b-8bac-049e0742acb5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:07.252386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389006e5-def2-4e4e-8c57-b3a1a8ea7c4c None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:07.252588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389006e5-def2-4e4e-8c57-b3a1a8ea7c4c None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:07.255405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 63/120] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:07.261353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21849ff7-6561-403b-82e0-7153a8372f33 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:07.261657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21849ff7-6561-403b-82e0-7153a8372f33 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:07.262540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21849ff7-6561-403b-82e0-7153a8372f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:07.268146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 58/121] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:38:07 2022] PUT /identity/v3/projects/130f64a6280e4b83bf885edf8986e895/users/0e508b7987364e59b4008c6b649a48e0/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:07.273411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493b133-f77e-469b-8bac-049e0742acb5 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:07.303563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-21849ff7-6561-403b-82e0-7153a8372f33 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:07.304771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-21849ff7-6561-403b-82e0-7153a8372f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:07.393352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:07.424186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:07.428209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 64/122] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:07.429179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21849ff7-6561-403b-82e0-7153a8372f33 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:07.463847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493b133-f77e-469b-8bac-049e0742acb5 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:07.538399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4996c-42e6-4d84-b478-9deb105a9a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:07.538983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4996c-42e6-4d84-b478-9deb105a9a87 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:07.539267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4996c-42e6-4d84-b478-9deb105a9a87 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:07.539539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4996c-42e6-4d84-b478-9deb105a9a87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:07.558444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4996c-42e6-4d84-b478-9deb105a9a87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-8fb4996c-42e6-4d84-b478-9deb105a9a87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:07.563896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:07.564745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:07.565520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 59/123] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:07.573381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4996c-42e6-4d84-b478-9deb105a9a87 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:07.628467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21849ff7-6561-403b-82e0-7153a8372f33 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:07.692923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dced7a-94a2-4070-b8cd-a30352968b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:07.694128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dced7a-94a2-4070-b8cd-a30352968b63 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:07.694613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dced7a-94a2-4070-b8cd-a30352968b63 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:07.695101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dced7a-94a2-4070-b8cd-a30352968b63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:07.702955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4996c-42e6-4d84-b478-9deb105a9a87 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:07.720040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dced7a-94a2-4070-b8cd-a30352968b63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-81dced7a-94a2-4070-b8cd-a30352968b63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:07.725550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:07.726387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:07.727162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:07.729591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 65/124] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:38:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:07.735712 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dced7a-94a2-4070-b8cd-a30352968b63 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:07.743398 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53281f-4aea-4d95-9eae-985c51a8381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:07.744029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53281f-4aea-4d95-9eae-985c51a8381d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:07.744304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53281f-4aea-4d95-9eae-985c51a8381d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:07.744581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53281f-4aea-4d95-9eae-985c51a8381d None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:07.745263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53281f-4aea-4d95-9eae-985c51a8381d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:07.747724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53281f-4aea-4d95-9eae-985c51a8381d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:07.752524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53281f-4aea-4d95-9eae-985c51a8381d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:07.765301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 60/125] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:07.770831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53281f-4aea-4d95-9eae-985c51a8381d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:07.777817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:07.827876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:07.894763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53281f-4aea-4d95-9eae-985c51a8381d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:07.934227 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c283c-5726-4c63-abaa-c817358065f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:07.935232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c283c-5726-4c63-abaa-c817358065f2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:07.935719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c283c-5726-4c63-abaa-c817358065f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:07.936193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c283c-5726-4c63-abaa-c817358065f2 None admin] PATH_INFO: `/v3/users/0e508b7987364e59b4008c6b649a48e0` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:07.939432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c283c-5726-4c63-abaa-c817358065f2 None admin] RBAC: Authorizing `identity:get_user(user_id=0e508b7987364e59b4008c6b649a48e0)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:07.941124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c283c-5726-4c63-abaa-c817358065f2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:07.945689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 61/126] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:07 2022] GET /identity/v3/users/0e508b7987364e59b4008c6b649a48e0 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:07.946710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c283c-5726-4c63-abaa-c817358065f2 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:07.970326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dced7a-94a2-4070-b8cd-a30352968b63 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:08.039338 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1565552-ed5b-4596-af13-c655268c47ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:08.040362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1565552-ed5b-4596-af13-c655268c47ca None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:08.041589 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1565552-ed5b-4596-af13-c655268c47ca None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:08.042065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1565552-ed5b-4596-af13-c655268c47ca None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:08.044909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1565552-ed5b-4596-af13-c655268c47ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:08.045428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1565552-ed5b-4596-af13-c655268c47ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:08.047084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1565552-ed5b-4596-af13-c655268c47ca None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:08.066983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 66/127] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:08.067979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1565552-ed5b-4596-af13-c655268c47ca None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:08.072369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:08.118828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c283c-5726-4c63-abaa-c817358065f2 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:08.162174 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35177ac8-8e43-45b1-bc45-b13c03f99655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:08.162819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177ac8-8e43-45b1-bc45-b13c03f99655 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:08.163150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177ac8-8e43-45b1-bc45-b13c03f99655 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:08.163428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177ac8-8e43-45b1-bc45-b13c03f99655 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:08.175541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35177ac8-8e43-45b1-bc45-b13c03f99655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-35177ac8-8e43-45b1-bc45-b13c03f99655 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:08.182025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:08.182875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:08.183807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 62/128] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:08.184444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35177ac8-8e43-45b1-bc45-b13c03f99655 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:08.304011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35177ac8-8e43-45b1-bc45-b13c03f99655 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:08.316330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1565552-ed5b-4596-af13-c655268c47ca None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:08.317928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:08.335776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:08.342922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0203207-5341-420c-9bc7-9e31f404af6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:08.343626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203207-5341-420c-9bc7-9e31f404af6e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:08.343919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203207-5341-420c-9bc7-9e31f404af6e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:08.344201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203207-5341-420c-9bc7-9e31f404af6e None admin] PATH_INFO: `/v3/users/dd9397aad94548f284667a53c0b499d2` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:08.345409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0203207-5341-420c-9bc7-9e31f404af6e None admin] RBAC: Authorizing `identity:get_user(user_id=dd9397aad94548f284667a53c0b499d2)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:08.347465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0203207-5341-420c-9bc7-9e31f404af6e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:08.349073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 63/129] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:08 2022] GET /identity/v3/users/dd9397aad94548f284667a53c0b499d2 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:08.350022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0203207-5341-420c-9bc7-9e31f404af6e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:08.397699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6b194-3f88-447e-ac2f-638a066c9a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:08.398267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6b194-3f88-447e-ac2f-638a066c9a27 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:08.398549 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6b194-3f88-447e-ac2f-638a066c9a27 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:08.398821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6b194-3f88-447e-ac2f-638a066c9a27 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:08.399973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6b194-3f88-447e-ac2f-638a066c9a27 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:08.401427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6b194-3f88-447e-ac2f-638a066c9a27 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:08.411195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 67/130] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:08 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:08.412006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6b194-3f88-447e-ac2f-638a066c9a27 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:08.464666 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0203207-5341-420c-9bc7-9e31f404af6e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:08.518303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed744c0-f359-4b3e-883c-3574110697f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:08.518931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed744c0-f359-4b3e-883c-3574110697f5 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:08.519210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed744c0-f359-4b3e-883c-3574110697f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:08.519483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed744c0-f359-4b3e-883c-3574110697f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:08.520162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed744c0-f359-4b3e-883c-3574110697f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:08.520470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed744c0-f359-4b3e-883c-3574110697f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:08.521895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed744c0-f359-4b3e-883c-3574110697f5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:08.544721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 64/131] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:08.545377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed744c0-f359-4b3e-883c-3574110697f5 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:08.628421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6b194-3f88-447e-ac2f-638a066c9a27 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:08.665444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543f56-1072-4988-96a2-b9692aee4573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:08.666783 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543f56-1072-4988-96a2-b9692aee4573 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:08.667244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543f56-1072-4988-96a2-b9692aee4573 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:08.667655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543f56-1072-4988-96a2-b9692aee4573 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:08.669191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543f56-1072-4988-96a2-b9692aee4573 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:08.670908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543f56-1072-4988-96a2-b9692aee4573 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:08.672942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 68/132] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:08 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:08.673890 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543f56-1072-4988-96a2-b9692aee4573 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:08.725859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:08.730308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:08.775196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed744c0-f359-4b3e-883c-3574110697f5 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:08.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543f56-1072-4988-96a2-b9692aee4573 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:08.811533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ee01a-fc71-4c6c-9771-4231e3a0b1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:08.812450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ee01a-fc71-4c6c-9771-4231e3a0b1c5 None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:08.812837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ee01a-fc71-4c6c-9771-4231e3a0b1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:08.813302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ee01a-fc71-4c6c-9771-4231e3a0b1c5 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895/users/0e508b7987364e59b4008c6b649a48e0/roles/8cd21d388193490bac3aa386bcc18763` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:08.815692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ee01a-fc71-4c6c-9771-4231e3a0b1c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=130f64a6280e4b83bf885edf8986e895, user_id=0e508b7987364e59b4008c6b649a48e0, role_id=8cd21d388193490bac3aa386bcc18763)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:08.817371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ee01a-fc71-4c6c-9771-4231e3a0b1c5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:08.836871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 65/133] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:38:08 2022] PUT /identity/v3/projects/130f64a6280e4b83bf885edf8986e895/users/0e508b7987364e59b4008c6b649a48e0/roles/8cd21d388193490bac3aa386bcc18763 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:08.838311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ee01a-fc71-4c6c-9771-4231e3a0b1c5 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:08.845253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a5658-bb8c-4c9b-86db-5bd389aed6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:08.846302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a5658-bb8c-4c9b-86db-5bd389aed6fd None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:08.846751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a5658-bb8c-4c9b-86db-5bd389aed6fd None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:08.847215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a5658-bb8c-4c9b-86db-5bd389aed6fd None admin] PATH_INFO: `/v3/users/0e508b7987364e59b4008c6b649a48e0` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:08.848676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a5658-bb8c-4c9b-86db-5bd389aed6fd None admin] RBAC: Authorizing `identity:get_user(user_id=0e508b7987364e59b4008c6b649a48e0)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:08.850501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a5658-bb8c-4c9b-86db-5bd389aed6fd None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:08.852437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 69/134] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:08 2022] GET /identity/v3/users/0e508b7987364e59b4008c6b649a48e0 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:08.858954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7f7a7-ad79-411c-8538-140d8b3cb458 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:08.859515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7f7a7-ad79-411c-8538-140d8b3cb458 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:08.859955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7f7a7-ad79-411c-8538-140d8b3cb458 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:08.861816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 70/135] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:08.863571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7f7a7-ad79-411c-8538-140d8b3cb458 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:08.958116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ee01a-fc71-4c6c-9771-4231e3a0b1c5 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:08.980823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:08.987622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7f7a7-ad79-411c-8538-140d8b3cb458 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:08.999093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:09.000396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ef7ac-e496-4b7b-84e7-30d2a29013dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:09.001242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ef7ac-e496-4b7b-84e7-30d2a29013dd None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:09.001719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ef7ac-e496-4b7b-84e7-30d2a29013dd None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:09.002201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ef7ac-e496-4b7b-84e7-30d2a29013dd None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895/users/dd9397aad94548f284667a53c0b499d2/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:09.004365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ef7ac-e496-4b7b-84e7-30d2a29013dd None admin] RBAC: Authorizing `identity:create_grant(project_id=130f64a6280e4b83bf885edf8986e895, user_id=dd9397aad94548f284667a53c0b499d2, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:09.006048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ef7ac-e496-4b7b-84e7-30d2a29013dd None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:09.030627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 66/136] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:38:08 2022] PUT /identity/v3/projects/130f64a6280e4b83bf885edf8986e895/users/dd9397aad94548f284667a53c0b499d2/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:09.032654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67438a30-2b35-4533-a27d-901cf6c4cfd7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:09.033224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67438a30-2b35-4533-a27d-901cf6c4cfd7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:09.033837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67438a30-2b35-4533-a27d-901cf6c4cfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:09.034748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1feadd-cd71-439b-9dd6-d42447d368a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:09.035463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1feadd-cd71-439b-9dd6-d42447d368a0 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:09.041532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1feadd-cd71-439b-9dd6-d42447d368a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:09.041532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1feadd-cd71-439b-9dd6-d42447d368a0 None admin] PATH_INFO: `/v3/projects/cd94cbfdffec47d597dde3e2bc2d86d5` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:09.042876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1feadd-cd71-439b-9dd6-d42447d368a0 None admin] RBAC: Authorizing `identity:get_project(project_id=cd94cbfdffec47d597dde3e2bc2d86d5)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:09.044816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1feadd-cd71-439b-9dd6-d42447d368a0 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:09.046851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 71/137] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:08 2022] GET /identity/v3/projects/cd94cbfdffec47d597dde3e2bc2d86d5 => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:09.052874 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1feadd-cd71-439b-9dd6-d42447d368a0 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:09.091251 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-67438a30-2b35-4533-a27d-901cf6c4cfd7 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:09.092496 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-67438a30-2b35-4533-a27d-901cf6c4cfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:09.198763 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1feadd-cd71-439b-9dd6-d42447d368a0 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:09.236011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 67/138] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:09.237759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1449d-a467-41f2-9f81-3a8acadb5294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:09.238429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1449d-a467-41f2-9f81-3a8acadb5294 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:09.238705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1449d-a467-41f2-9f81-3a8acadb5294 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:09.238975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1449d-a467-41f2-9f81-3a8acadb5294 None admin] PATH_INFO: `/v3/projects/cd94cbfdffec47d597dde3e2bc2d86d5/users/0e508b7987364e59b4008c6b649a48e0/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:09.241042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1449d-a467-41f2-9f81-3a8acadb5294 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd94cbfdffec47d597dde3e2bc2d86d5, user_id=0e508b7987364e59b4008c6b649a48e0, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:09.243195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1449d-a467-41f2-9f81-3a8acadb5294 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:09.263274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 72/139] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:38:09 2022] PUT /identity/v3/projects/cd94cbfdffec47d597dde3e2bc2d86d5/users/0e508b7987364e59b4008c6b649a48e0/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:38:09.609254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67438a30-2b35-4533-a27d-901cf6c4cfd7 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:09.725740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67438a30-2b35-4533-a27d-901cf6c4cfd7 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:09.743262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:09.775556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06acc7a-4e08-4265-a140-7b60ad938aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:09.776204 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06acc7a-4e08-4265-a140-7b60ad938aab None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:09.776485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06acc7a-4e08-4265-a140-7b60ad938aab None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:09.776753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06acc7a-4e08-4265-a140-7b60ad938aab None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:09.779438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06acc7a-4e08-4265-a140-7b60ad938aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:09.780908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06acc7a-4e08-4265-a140-7b60ad938aab None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:09.782752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 68/140] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:09.784454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40394fc-21b2-459d-b620-53276da6a50a None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:09.784896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40394fc-21b2-459d-b620-53276da6a50a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:09.785001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40394fc-21b2-459d-b620-53276da6a50a None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:09.786357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 73/141] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:09.788296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06acc7a-4e08-4265-a140-7b60ad938aab None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:09.803599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be769c4-dd88-4bd4-880d-3a62caae8d27 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:09.803881 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be769c4-dd88-4bd4-880d-3a62caae8d27 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:09.804144 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be769c4-dd88-4bd4-880d-3a62caae8d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:09.836425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-0be769c4-dd88-4bd4-880d-3a62caae8d27 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:09.837670 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-0be769c4-dd88-4bd4-880d-3a62caae8d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:09.888616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:09.953220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06acc7a-4e08-4265-a140-7b60ad938aab None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:09.959629 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 74/142] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:10.005961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff20fc-0f1f-41b8-83d4-98cafe8c4b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:10.006578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff20fc-0f1f-41b8-83d4-98cafe8c4b29 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:10.006859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff20fc-0f1f-41b8-83d4-98cafe8c4b29 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:10.007131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff20fc-0f1f-41b8-83d4-98cafe8c4b29 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:10.007797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff20fc-0f1f-41b8-83d4-98cafe8c4b29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:10.009240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff20fc-0f1f-41b8-83d4-98cafe8c4b29 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:10.040723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 69/143] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:38:09 2022] POST /identity/v3/users => generated 283 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:38:10.291078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a186e-52f3-425d-ab59-1adfe047085b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:10.291438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a186e-52f3-425d-ab59-1adfe047085b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:10.291610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a186e-52f3-425d-ab59-1adfe047085b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:10.293160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 75/144] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:10.298713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4716-08cc-49c9-a0f2-c3638a025647 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:10.298998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4716-08cc-49c9-a0f2-c3638a025647 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:10.299274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4716-08cc-49c9-a0f2-c3638a025647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:10.327921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6fbb4716-08cc-49c9-a0f2-c3638a025647 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:10.329127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6fbb4716-08cc-49c9-a0f2-c3638a025647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:10.367527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:10.421926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 70/145] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:10.613204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a186e-52f3-425d-ab59-1adfe047085b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:10.729343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a186e-52f3-425d-ab59-1adfe047085b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:10.773154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d46cb-c811-4a07-88a9-5d79b6116509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:10.773816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d46cb-c811-4a07-88a9-5d79b6116509 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:10.774094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d46cb-c811-4a07-88a9-5d79b6116509 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:10.774363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d46cb-c811-4a07-88a9-5d79b6116509 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:10.786076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d46cb-c811-4a07-88a9-5d79b6116509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-e38d46cb-c811-4a07-88a9-5d79b6116509 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:10.790966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:10.791847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:10.792600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 76/146] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:10.799423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb4716-08cc-49c9-a0f2-c3638a025647 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:10.864233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d46cb-c811-4a07-88a9-5d79b6116509 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:10.901475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:10.962187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb4716-08cc-49c9-a0f2-c3638a025647 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:11.012854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6682e-f8df-4fb0-bec1-e3f651bfbfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:11.013936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6682e-f8df-4fb0-bec1-e3f651bfbfb2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:11.014218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6682e-f8df-4fb0-bec1-e3f651bfbfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:11.014487 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6682e-f8df-4fb0-bec1-e3f651bfbfb2 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:11.015305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6682e-f8df-4fb0-bec1-e3f651bfbfb2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:11.015631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6682e-f8df-4fb0-bec1-e3f651bfbfb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:11.017229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6682e-f8df-4fb0-bec1-e3f651bfbfb2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:11.024307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d46cb-c811-4a07-88a9-5d79b6116509 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:11.028623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 71/147] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:11.037585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6682e-f8df-4fb0-bec1-e3f651bfbfb2 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:11.079637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:11.091098 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbdbb8-103e-44ad-9c2f-98e349d2ea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:11.091776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbdbb8-103e-44ad-9c2f-98e349d2ea4d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:11.092066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbdbb8-103e-44ad-9c2f-98e349d2ea4d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:11.092331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbdbb8-103e-44ad-9c2f-98e349d2ea4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:11.114578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbdbb8-103e-44ad-9c2f-98e349d2ea4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-94fbdbb8-103e-44ad-9c2f-98e349d2ea4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:11.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:11.121058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:11.121852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:11.124184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 77/148] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:11.129976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbdbb8-103e-44ad-9c2f-98e349d2ea4d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:11.203847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6682e-f8df-4fb0-bec1-e3f651bfbfb2 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:11.246848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:11.252043 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83344ee6-ee6a-4d0a-b60b-1ca828a8f500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:11.252885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344ee6-ee6a-4d0a-b60b-1ca828a8f500 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:11.253313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344ee6-ee6a-4d0a-b60b-1ca828a8f500 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:11.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344ee6-ee6a-4d0a-b60b-1ca828a8f500 None admin] PATH_INFO: `/v3/users/780865e68bbc4cc49315105a3c0b90f7` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:11.255117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344ee6-ee6a-4d0a-b60b-1ca828a8f500 None admin] RBAC: Authorizing `identity:get_user(user_id=780865e68bbc4cc49315105a3c0b90f7)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:11.256709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344ee6-ee6a-4d0a-b60b-1ca828a8f500 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:11.258500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 72/149] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:11 2022] GET /identity/v3/users/780865e68bbc4cc49315105a3c0b90f7 => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:11.264739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83344ee6-ee6a-4d0a-b60b-1ca828a8f500 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:11.271116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbdbb8-103e-44ad-9c2f-98e349d2ea4d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:11.320799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4b5ac-d08a-4657-803d-e39cebbd9d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:11.322045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4b5ac-d08a-4657-803d-e39cebbd9d31 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:11.323016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4b5ac-d08a-4657-803d-e39cebbd9d31 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:11.323781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4b5ac-d08a-4657-803d-e39cebbd9d31 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:11.325554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4b5ac-d08a-4657-803d-e39cebbd9d31 None admin] RBAC: Adding query filter params (name=member) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:11.326090 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4b5ac-d08a-4657-803d-e39cebbd9d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:11.328088 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4b5ac-d08a-4657-803d-e39cebbd9d31 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:11.340012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 78/150] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:11.345826 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4b5ac-d08a-4657-803d-e39cebbd9d31 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:11.409564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:11.426527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83344ee6-ee6a-4d0a-b60b-1ca828a8f500 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:11.500425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f6014-a02d-45f6-a046-821555be4ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:11.501444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6014-a02d-45f6-a046-821555be4ad8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:11.502004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6014-a02d-45f6-a046-821555be4ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:11.503835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6014-a02d-45f6-a046-821555be4ad8 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:11.506055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f6014-a02d-45f6-a046-821555be4ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:11.507749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f6014-a02d-45f6-a046-821555be4ad8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:11.510765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 73/151] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:11 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:11.517238 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f6014-a02d-45f6-a046-821555be4ad8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:11.539966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4b5ac-d08a-4657-803d-e39cebbd9d31 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:11.587921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2ff84-c47e-440d-b35d-455a3b3a1a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:11.591083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2ff84-c47e-440d-b35d-455a3b3a1a56 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:11.591583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2ff84-c47e-440d-b35d-455a3b3a1a56 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:11.592072 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2ff84-c47e-440d-b35d-455a3b3a1a56 None admin] PATH_INFO: `/v3/users/0e508b7987364e59b4008c6b649a48e0` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:11.595052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2ff84-c47e-440d-b35d-455a3b3a1a56 None admin] RBAC: Authorizing `identity:get_user(user_id=0e508b7987364e59b4008c6b649a48e0)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:11.596752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2ff84-c47e-440d-b35d-455a3b3a1a56 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:11.599676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 79/152] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:11 2022] GET /identity/v3/users/0e508b7987364e59b4008c6b649a48e0 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:11.604739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2ff84-c47e-440d-b35d-455a3b3a1a56 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:11.626906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:11.687519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f6014-a02d-45f6-a046-821555be4ad8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:11.731877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2ff84-c47e-440d-b35d-455a3b3a1a56 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:11.774067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354b10f-10a1-4365-aa9e-69f788ad410f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:11.774469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7ccf7-9c18-428a-9ac3-4d29de5af402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:11.774743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354b10f-10a1-4365-aa9e-69f788ad410f None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:11.775026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354b10f-10a1-4365-aa9e-69f788ad410f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:11.775299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354b10f-10a1-4365-aa9e-69f788ad410f None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895/users/780865e68bbc4cc49315105a3c0b90f7/roles/8d46737ab3ff453a800fe6b47805eb5a` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:11.775720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7ccf7-9c18-428a-9ac3-4d29de5af402 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:11.776136 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7ccf7-9c18-428a-9ac3-4d29de5af402 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:11.776564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7ccf7-9c18-428a-9ac3-4d29de5af402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:11.777394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354b10f-10a1-4365-aa9e-69f788ad410f None admin] RBAC: Authorizing `identity:create_grant(project_id=130f64a6280e4b83bf885edf8986e895, user_id=780865e68bbc4cc49315105a3c0b90f7, role_id=8d46737ab3ff453a800fe6b47805eb5a)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:11.778883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354b10f-10a1-4365-aa9e-69f788ad410f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:11.821994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 74/153] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:38:11 2022] PUT /identity/v3/projects/130f64a6280e4b83bf885edf8986e895/users/780865e68bbc4cc49315105a3c0b90f7/roles/8d46737ab3ff453a800fe6b47805eb5a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:11.866481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7ccf7-9c18-428a-9ac3-4d29de5af402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:11.868856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7ccf7-9c18-428a-9ac3-4d29de5af402 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:11.897219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:12.018702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 80/154] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 14:38:12.024364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354b10f-10a1-4365-aa9e-69f788ad410f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:12.064985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:12.149332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a88e1d-e1eb-42b9-b2c9-c78307ceb242 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:12.149877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a88e1d-e1eb-42b9-b2c9-c78307ceb242 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:12.150398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a88e1d-e1eb-42b9-b2c9-c78307ceb242 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:12.154114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 81/155] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:12.159712 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e03e02-edd9-47dd-a058-39fc36afe2ba None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:12.160192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e03e02-edd9-47dd-a058-39fc36afe2ba None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:12.160482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354b10f-10a1-4365-aa9e-69f788ad410f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:12.160784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e03e02-edd9-47dd-a058-39fc36afe2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:12.197691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c2e03e02-edd9-47dd-a058-39fc36afe2ba None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:12.199211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c2e03e02-edd9-47dd-a058-39fc36afe2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:12.231311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea39b7-2e41-4159-83a9-68a987936855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:12.231974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea39b7-2e41-4159-83a9-68a987936855 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:12.232256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea39b7-2e41-4159-83a9-68a987936855 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:12.232531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea39b7-2e41-4159-83a9-68a987936855 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:12.233710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea39b7-2e41-4159-83a9-68a987936855 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:12.237016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea39b7-2e41-4159-83a9-68a987936855 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:12.237016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 75/156] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:12 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:12.243087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea39b7-2e41-4159-83a9-68a987936855 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:12.353984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 82/157] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:12.418466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:12.422230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea39b7-2e41-4159-83a9-68a987936855 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:12.465159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70707c2-5de0-4cc6-b59d-d59a835e4abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:12.465804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70707c2-5de0-4cc6-b59d-d59a835e4abd None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:12.466088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70707c2-5de0-4cc6-b59d-d59a835e4abd None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:12.466361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70707c2-5de0-4cc6-b59d-d59a835e4abd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:12.467102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70707c2-5de0-4cc6-b59d-d59a835e4abd None admin] RBAC: Adding query filter params (role.id=47112770ffdc45638da090c45073888e, scope.project.id=130f64a6280e4b83bf885edf8986e895, user.id=0e508b7987364e59b4008c6b649a48e0) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:12.467412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70707c2-5de0-4cc6-b59d-d59a835e4abd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:12.468887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70707c2-5de0-4cc6-b59d-d59a835e4abd None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:12.475087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 76/158] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:12 2022] GET /identity/v3/role_assignments?user.id=0e508b7987364e59b4008c6b649a48e0&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=47112770ffdc45638da090c45073888e => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:12.559624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be648d4-6e45-4dde-806b-ac8f4697cecc None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:12.559921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be648d4-6e45-4dde-806b-ac8f4697cecc None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:12.560104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be648d4-6e45-4dde-806b-ac8f4697cecc None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:12.561737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 83/159] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:12.567147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed2614-39be-43a0-978a-5bbe671c6d93 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:12.567434 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed2614-39be-43a0-978a-5bbe671c6d93 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:12.567709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed2614-39be-43a0-978a-5bbe671c6d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:12.592002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-84ed2614-39be-43a0-978a-5bbe671c6d93 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:12.593218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-84ed2614-39be-43a0-978a-5bbe671c6d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:12.683898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 77/160] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:13.013554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be648d4-6e45-4dde-806b-ac8f4697cecc None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:13.014074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944ab5c-9f30-4bea-b973-3bac5f8efa3f None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:13.014433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944ab5c-9f30-4bea-b973-3bac5f8efa3f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:13.014744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944ab5c-9f30-4bea-b973-3bac5f8efa3f None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:13.016292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 78/161] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:13.025526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860edac-4bae-4042-9d76-f01f2dfa1d82 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:13.025526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860edac-4bae-4042-9d76-f01f2dfa1d82 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:13.025526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860edac-4bae-4042-9d76-f01f2dfa1d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:13.046016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-5860edac-4bae-4042-9d76-f01f2dfa1d82 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:13.047227 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5860edac-4bae-4042-9d76-f01f2dfa1d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:13.085965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:13.172696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 79/162] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:13.182817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be648d4-6e45-4dde-806b-ac8f4697cecc None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:13.230154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004d465-342e-469c-bca0-d55eccba4ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:13.231133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d465-342e-469c-bca0-d55eccba4ad7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:13.231570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d465-342e-469c-bca0-d55eccba4ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:13.231995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d465-342e-469c-bca0-d55eccba4ad7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:13.246994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004d465-342e-469c-bca0-d55eccba4ad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-c004d465-342e-469c-bca0-d55eccba4ad7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:13.252265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:13.253316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:13.254335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:13.256771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 84/163] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:13.263073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860edac-4bae-4042-9d76-f01f2dfa1d82 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:13.296238 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:13.308119 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004d465-342e-469c-bca0-d55eccba4ad7 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:13.448546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860edac-4bae-4042-9d76-f01f2dfa1d82 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:13.466564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004d465-342e-469c-bca0-d55eccba4ad7 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:13.469408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:13.513072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92b731-375c-45df-ac8e-876e277e92df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:13.513714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92b731-375c-45df-ac8e-876e277e92df None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:13.514447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92b731-375c-45df-ac8e-876e277e92df None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:13.514447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92b731-375c-45df-ac8e-876e277e92df None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:13.514970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92b731-375c-45df-ac8e-876e277e92df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:13.515274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92b731-375c-45df-ac8e-876e277e92df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:13.516667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92b731-375c-45df-ac8e-876e277e92df None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:13.531618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 80/164] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:13.537581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92b731-375c-45df-ac8e-876e277e92df None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:13.548927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb6b26-46a9-433c-8699-c698b1c1c7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:13.549511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb6b26-46a9-433c-8699-c698b1c1c7d2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:13.549788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb6b26-46a9-433c-8699-c698b1c1c7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:13.550054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb6b26-46a9-433c-8699-c698b1c1c7d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:13.582440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb6b26-46a9-433c-8699-c698b1c1c7d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-92cb6b26-46a9-433c-8699-c698b1c1c7d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:13.587331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:13.588200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:13.588972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 85/165] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:38:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:13.594989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb6b26-46a9-433c-8699-c698b1c1c7d2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:13.697181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:13.754545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92b731-375c-45df-ac8e-876e277e92df None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:13.790693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb6b26-46a9-433c-8699-c698b1c1c7d2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:13.844395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e2d25-455f-48eb-af42-29c46a53adb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:13.845288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e2d25-455f-48eb-af42-29c46a53adb1 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:13.845753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e2d25-455f-48eb-af42-29c46a53adb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:13.846163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e2d25-455f-48eb-af42-29c46a53adb1 None admin] PATH_INFO: `/v3/users/0e508b7987364e59b4008c6b649a48e0` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:13.850205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e2d25-455f-48eb-af42-29c46a53adb1 None admin] RBAC: Authorizing `identity:get_user(user_id=0e508b7987364e59b4008c6b649a48e0)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:13.851791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e2d25-455f-48eb-af42-29c46a53adb1 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:13.853542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 81/166] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:13 2022] GET /identity/v3/users/0e508b7987364e59b4008c6b649a48e0 => generated 275 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:13.854486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e2d25-455f-48eb-af42-29c46a53adb1 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:13.855735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48625d-4fff-4503-931a-76cb20f7c76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:13.856538 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48625d-4fff-4503-931a-76cb20f7c76d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:13.856968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48625d-4fff-4503-931a-76cb20f7c76d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:13.857388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48625d-4fff-4503-931a-76cb20f7c76d None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:13.858418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48625d-4fff-4503-931a-76cb20f7c76d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:13.858885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48625d-4fff-4503-931a-76cb20f7c76d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:13.860438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48625d-4fff-4503-931a-76cb20f7c76d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:13.875374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 86/167] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:13.876580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48625d-4fff-4503-931a-76cb20f7c76d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:13.928410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:14.045870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e2d25-455f-48eb-af42-29c46a53adb1 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:14.049217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48625d-4fff-4503-931a-76cb20f7c76d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:14.111614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ccea8-325b-413e-bb63-15ff39cfd105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:14.111806 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37e8bb-f918-40aa-bc23-10fa2b4237bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:14.112177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e8bb-f918-40aa-bc23-10fa2b4237bf None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:14.112460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e8bb-f918-40aa-bc23-10fa2b4237bf None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:14.112734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e8bb-f918-40aa-bc23-10fa2b4237bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:14.113303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ccea8-325b-413e-bb63-15ff39cfd105 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:14.113789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ccea8-325b-413e-bb63-15ff39cfd105 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:14.114262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ccea8-325b-413e-bb63-15ff39cfd105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:14.129102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37e8bb-f918-40aa-bc23-10fa2b4237bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-6d37e8bb-f918-40aa-bc23-10fa2b4237bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:14.134389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:14.135321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:14.136078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:14.141870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 82/168] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:14.142697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37e8bb-f918-40aa-bc23-10fa2b4237bf None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:14.158015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:14.254586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37e8bb-f918-40aa-bc23-10fa2b4237bf None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:14.258500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ccea8-325b-413e-bb63-15ff39cfd105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:14.260013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ccea8-325b-413e-bb63-15ff39cfd105 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:14.299118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6f7b7-0f36-4188-ae56-781dc9396fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:14.299968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6f7b7-0f36-4188-ae56-781dc9396fd8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:14.300393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6f7b7-0f36-4188-ae56-781dc9396fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:14.300830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6f7b7-0f36-4188-ae56-781dc9396fd8 None admin] PATH_INFO: `/v3/users/dd9397aad94548f284667a53c0b499d2` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:14.302154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6f7b7-0f36-4188-ae56-781dc9396fd8 None admin] RBAC: Authorizing `identity:get_user(user_id=dd9397aad94548f284667a53c0b499d2)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:14.303766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6f7b7-0f36-4188-ae56-781dc9396fd8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:14.305838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 83/169] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:14 2022] GET /identity/v3/users/dd9397aad94548f284667a53c0b499d2 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:14.306765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6f7b7-0f36-4188-ae56-781dc9396fd8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:14.315882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:14.388291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:14.415021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6f7b7-0f36-4188-ae56-781dc9396fd8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:14.451967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a43a4b-f1e7-4e57-95c2-20d464472385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:14.452822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a43a4b-f1e7-4e57-95c2-20d464472385 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:14.453247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a43a4b-f1e7-4e57-95c2-20d464472385 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:14.453935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a43a4b-f1e7-4e57-95c2-20d464472385 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:14.454860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a43a4b-f1e7-4e57-95c2-20d464472385 None admin] RBAC: Adding query filter params (name=member) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:14.455319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a43a4b-f1e7-4e57-95c2-20d464472385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:14.456837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a43a4b-f1e7-4e57-95c2-20d464472385 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:14.466712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 84/170] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:14.467823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a43a4b-f1e7-4e57-95c2-20d464472385 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:14.472756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 87/171] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:14.473621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ccea8-325b-413e-bb63-15ff39cfd105 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:14.597982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ccea8-325b-413e-bb63-15ff39cfd105 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:14.624340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:14.632920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d1411-704f-4c2b-9c75-1d5ed98743f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:14.633574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d1411-704f-4c2b-9c75-1d5ed98743f2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:14.633854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d1411-704f-4c2b-9c75-1d5ed98743f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:14.634362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d1411-704f-4c2b-9c75-1d5ed98743f2 None admin] PATH_INFO: `/v3/users/0e508b7987364e59b4008c6b649a48e0` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:14.635512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d1411-704f-4c2b-9c75-1d5ed98743f2 None admin] RBAC: Authorizing `identity:get_user(user_id=0e508b7987364e59b4008c6b649a48e0)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:14.635910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a43a4b-f1e7-4e57-95c2-20d464472385 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:14.637224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d1411-704f-4c2b-9c75-1d5ed98743f2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:14.643732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 88/172] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:14 2022] GET /identity/v3/users/0e508b7987364e59b4008c6b649a48e0 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:14.644621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d1411-704f-4c2b-9c75-1d5ed98743f2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:14.672555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041900dc-184a-40e3-8a16-c15de3cb959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:14.673520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041900dc-184a-40e3-8a16-c15de3cb959f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:14.673944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041900dc-184a-40e3-8a16-c15de3cb959f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:14.674388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041900dc-184a-40e3-8a16-c15de3cb959f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:14.678748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:14.746113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041900dc-184a-40e3-8a16-c15de3cb959f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:14.748049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041900dc-184a-40e3-8a16-c15de3cb959f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:14.810311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d1411-704f-4c2b-9c75-1d5ed98743f2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:14.851588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 85/173] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:14.852950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041900dc-184a-40e3-8a16-c15de3cb959f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:14.870362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3afbd6-9331-429d-abf2-eedd62965fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:14.871622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3afbd6-9331-429d-abf2-eedd62965fe3 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:14.871898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3afbd6-9331-429d-abf2-eedd62965fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:14.872167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3afbd6-9331-429d-abf2-eedd62965fe3 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:14.873320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3afbd6-9331-429d-abf2-eedd62965fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:14.874798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3afbd6-9331-429d-abf2-eedd62965fe3 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:14.876423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 89/174] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:14 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:14.877347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3afbd6-9331-429d-abf2-eedd62965fe3 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:14.901967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:14.965573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041900dc-184a-40e3-8a16-c15de3cb959f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:14.977524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:15.005799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89c116-e653-4591-b544-8aa6ef21951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:15.006815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89c116-e653-4591-b544-8aa6ef21951f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.007325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89c116-e653-4591-b544-8aa6ef21951f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.007909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89c116-e653-4591-b544-8aa6ef21951f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.049073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3afbd6-9331-429d-abf2-eedd62965fe3 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:15.100738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89c116-e653-4591-b544-8aa6ef21951f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:15.103616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89c116-e653-4591-b544-8aa6ef21951f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:15.124632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585559ec-9999-45dc-b26a-df925970bd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:15.125265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585559ec-9999-45dc-b26a-df925970bd89 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.125746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585559ec-9999-45dc-b26a-df925970bd89 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.126162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585559ec-9999-45dc-b26a-df925970bd89 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.127454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585559ec-9999-45dc-b26a-df925970bd89 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:15.129062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585559ec-9999-45dc-b26a-df925970bd89 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:15.131249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 90/175] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:14 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:15.132180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585559ec-9999-45dc-b26a-df925970bd89 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:15.153107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:15.228782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 86/176] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:15.230484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a25dd7-9533-4a12-b386-6ed927283485 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.230984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a25dd7-9533-4a12-b386-6ed927283485 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.231470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a25dd7-9533-4a12-b386-6ed927283485 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.233038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 87/177] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:15.233965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a25dd7-9533-4a12-b386-6ed927283485 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:15.301184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585559ec-9999-45dc-b26a-df925970bd89 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:15.329702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:15.335936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fedca-cfab-42da-adf4-08263bf2c29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:15.336755 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fedca-cfab-42da-adf4-08263bf2c29f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.337314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fedca-cfab-42da-adf4-08263bf2c29f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.337765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fedca-cfab-42da-adf4-08263bf2c29f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.338628 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fedca-cfab-42da-adf4-08263bf2c29f None admin] RBAC: Adding query filter params (role.id=8cd21d388193490bac3aa386bcc18763, scope.project.id=130f64a6280e4b83bf885edf8986e895, user.id=0e508b7987364e59b4008c6b649a48e0) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:15.339085 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fedca-cfab-42da-adf4-08263bf2c29f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:15.341201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fedca-cfab-42da-adf4-08263bf2c29f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:15.347210 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 91/178] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:15 2022] GET /identity/v3/role_assignments?user.id=0e508b7987364e59b4008c6b649a48e0&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=8cd21d388193490bac3aa386bcc18763 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:15.348166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fedca-cfab-42da-adf4-08263bf2c29f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:15.377374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a25dd7-9533-4a12-b386-6ed927283485 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:15.417763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fa6bf-5619-48f2-87a5-83f4d812a1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:15.418651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fa6bf-5619-48f2-87a5-83f4d812a1f4 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.419081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fa6bf-5619-48f2-87a5-83f4d812a1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.419620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fa6bf-5619-48f2-87a5-83f4d812a1f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.420612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fa6bf-5619-48f2-87a5-83f4d812a1f4 None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=130f64a6280e4b83bf885edf8986e895, user.id=dd9397aad94548f284667a53c0b499d2) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:15.421115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fa6bf-5619-48f2-87a5-83f4d812a1f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:15.423093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fa6bf-5619-48f2-87a5-83f4d812a1f4 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:15.425317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:15.434352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 88/179] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:15 2022] GET /identity/v3/role_assignments?user.id=dd9397aad94548f284667a53c0b499d2&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:15.436055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebd1a2-5169-474d-b8e6-44c02fac2946 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.436490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebd1a2-5169-474d-b8e6-44c02fac2946 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.436924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebd1a2-5169-474d-b8e6-44c02fac2946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.461683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fedca-cfab-42da-adf4-08263bf2c29f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:15.470973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d5ebd1a2-5169-474d-b8e6-44c02fac2946 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:15.472260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d5ebd1a2-5169-474d-b8e6-44c02fac2946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:15.501671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7888d5e-b442-4b0b-922c-1c00c3ccb973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:15.502727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7888d5e-b442-4b0b-922c-1c00c3ccb973 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.503192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7888d5e-b442-4b0b-922c-1c00c3ccb973 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.503617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7888d5e-b442-4b0b-922c-1c00c3ccb973 None admin] PATH_INFO: `/v3/projects/cd94cbfdffec47d597dde3e2bc2d86d5` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.504937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7888d5e-b442-4b0b-922c-1c00c3ccb973 None admin] RBAC: Authorizing `identity:get_project(project_id=cd94cbfdffec47d597dde3e2bc2d86d5)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:15.506565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7888d5e-b442-4b0b-922c-1c00c3ccb973 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:15.508696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 92/180] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:15 2022] GET /identity/v3/projects/cd94cbfdffec47d597dde3e2bc2d86d5 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:15.510087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75dab25-ca97-4be1-a725-01a2caf86548 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.510529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75dab25-ca97-4be1-a725-01a2caf86548 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.510978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75dab25-ca97-4be1-a725-01a2caf86548 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.512892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 93/181] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:15.523022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39623b7-6591-43af-9ce6-0c40cecbce85 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.523575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39623b7-6591-43af-9ce6-0c40cecbce85 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.524044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39623b7-6591-43af-9ce6-0c40cecbce85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.555518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-a39623b7-6591-43af-9ce6-0c40cecbce85 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:15.557047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a39623b7-6591-43af-9ce6-0c40cecbce85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:15.574627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 89/182] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:15.575691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebd1a2-5169-474d-b8e6-44c02fac2946 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:15.669582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:15.679694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 94/183] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:15.681051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04d77a-efe2-4f7e-8588-b546555828f8 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.681617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04d77a-efe2-4f7e-8588-b546555828f8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.682068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04d77a-efe2-4f7e-8588-b546555828f8 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.684166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 95/184] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:15.693969 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13371622-b62d-4aa3-813c-affe77890601 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.694535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13371622-b62d-4aa3-813c-affe77890601 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.694979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13371622-b62d-4aa3-813c-affe77890601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.718924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-13371622-b62d-4aa3-813c-affe77890601 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:15.720460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-13371622-b62d-4aa3-813c-affe77890601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:15.793417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 96/185] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:15.829058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebd1a2-5169-474d-b8e6-44c02fac2946 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:15.867889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:15.883072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6d1f4-6ba1-4b98-b738-e7b6f3b02d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:15.883683 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6d1f4-6ba1-4b98-b738-e7b6f3b02d9e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:15.883960 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6d1f4-6ba1-4b98-b738-e7b6f3b02d9e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:15.884228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6d1f4-6ba1-4b98-b738-e7b6f3b02d9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:15.884961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6d1f4-6ba1-4b98-b738-e7b6f3b02d9e None admin] RBAC: Adding query filter params (role.id=47112770ffdc45638da090c45073888e, scope.project.id=cd94cbfdffec47d597dde3e2bc2d86d5, user.id=0e508b7987364e59b4008c6b649a48e0) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:15.885261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6d1f4-6ba1-4b98-b738-e7b6f3b02d9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:15.886792 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6d1f4-6ba1-4b98-b738-e7b6f3b02d9e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:15.893929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 90/186] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:15 2022] GET /identity/v3/role_assignments?user.id=0e508b7987364e59b4008c6b649a48e0&scope.project.id=cd94cbfdffec47d597dde3e2bc2d86d5&role.id=47112770ffdc45638da090c45073888e => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:15.989829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13371622-b62d-4aa3-813c-affe77890601 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:16.104815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13371622-b62d-4aa3-813c-affe77890601 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:16.130588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:16.149010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6d1f4-6ba1-4b98-b738-e7b6f3b02d9e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:16.150018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802f344-e9ad-4eb9-accf-99fc75e3d290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:16.150659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802f344-e9ad-4eb9-accf-99fc75e3d290 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.150938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802f344-e9ad-4eb9-accf-99fc75e3d290 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.151203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802f344-e9ad-4eb9-accf-99fc75e3d290 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.165266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802f344-e9ad-4eb9-accf-99fc75e3d290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-f802f344-e9ad-4eb9-accf-99fc75e3d290 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:16.170410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:16.171319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:16.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 97/187] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:16.174933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df249e-683d-489f-a1cf-79688cdcd325 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.175771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df249e-683d-489f-a1cf-79688cdcd325 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.176052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df249e-683d-489f-a1cf-79688cdcd325 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.177577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 98/188] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:16.181672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df249e-683d-489f-a1cf-79688cdcd325 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:16.301040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df249e-683d-489f-a1cf-79688cdcd325 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:16.314350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6d1f4-6ba1-4b98-b738-e7b6f3b02d9e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:16.336462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563608a-ea1c-4b15-904d-3602930317b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:16.337078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563608a-ea1c-4b15-904d-3602930317b3 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.337355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563608a-ea1c-4b15-904d-3602930317b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.337657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563608a-ea1c-4b15-904d-3602930317b3 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.338325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563608a-ea1c-4b15-904d-3602930317b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:16.338627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563608a-ea1c-4b15-904d-3602930317b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:16.340050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563608a-ea1c-4b15-904d-3602930317b3 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:16.347924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 99/189] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:16.352662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c219dc-ada0-4a97-8584-1445cf5a8e19 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.352662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c219dc-ada0-4a97-8584-1445cf5a8e19 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.352662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c219dc-ada0-4a97-8584-1445cf5a8e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.353770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371b8e6-6664-4559-8c96-526730a339f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:16.354406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371b8e6-6664-4559-8c96-526730a339f5 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.354686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371b8e6-6664-4559-8c96-526730a339f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.355489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371b8e6-6664-4559-8c96-526730a339f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.356593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371b8e6-6664-4559-8c96-526730a339f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:16.358064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371b8e6-6664-4559-8c96-526730a339f5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:16.359615 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 91/190] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:16.360349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371b8e6-6664-4559-8c96-526730a339f5 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:16.370120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-06c219dc-ada0-4a97-8584-1445cf5a8e19 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:16.371888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-06c219dc-ada0-4a97-8584-1445cf5a8e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:16.411480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:16.418059 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:16.438482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 100/191] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:16.439288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c219dc-ada0-4a97-8584-1445cf5a8e19 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:16.465395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371b8e6-6664-4559-8c96-526730a339f5 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:16.501386 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e003997-6da4-4e35-9aa8-81ee2df351f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:16.502038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e003997-6da4-4e35-9aa8-81ee2df351f4 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.502312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e003997-6da4-4e35-9aa8-81ee2df351f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.502577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e003997-6da4-4e35-9aa8-81ee2df351f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.513151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e003997-6da4-4e35-9aa8-81ee2df351f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-8e003997-6da4-4e35-9aa8-81ee2df351f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:16.518484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:16.519336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:16.520110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 92/192] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:38:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:16.520784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e003997-6da4-4e35-9aa8-81ee2df351f4 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:16.556229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c219dc-ada0-4a97-8584-1445cf5a8e19 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:16.592627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9033f-3d0f-4467-b868-2df03d75f4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:16.593357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9033f-3d0f-4467-b868-2df03d75f4b4 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.593772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9033f-3d0f-4467-b868-2df03d75f4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.594074 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9033f-3d0f-4467-b868-2df03d75f4b4 None admin] PATH_INFO: `/v3/users/249e370f154d4748a72764c0aa73580e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.607769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9033f-3d0f-4467-b868-2df03d75f4b4 None admin] RBAC: Authorizing `identity:get_user(user_id=249e370f154d4748a72764c0aa73580e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:16.609458 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9033f-3d0f-4467-b868-2df03d75f4b4 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:16.615619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 101/193] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:16 2022] GET /identity/v3/users/249e370f154d4748a72764c0aa73580e => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:16.615619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9033f-3d0f-4467-b868-2df03d75f4b4 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:16.629376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e003997-6da4-4e35-9aa8-81ee2df351f4 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:16.645770 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:16.650834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:16.668027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cd2ac-a7f2-49c4-836a-a044a4b33c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:16.668652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cd2ac-a7f2-49c4-836a-a044a4b33c1d None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.668932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cd2ac-a7f2-49c4-836a-a044a4b33c1d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.669203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cd2ac-a7f2-49c4-836a-a044a4b33c1d None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.669914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cd2ac-a7f2-49c4-836a-a044a4b33c1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:16.671795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cd2ac-a7f2-49c4-836a-a044a4b33c1d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:16.693469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 93/194] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:38:16 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:38:16.694311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cd2ac-a7f2-49c4-836a-a044a4b33c1d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:16.737894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9033f-3d0f-4467-b868-2df03d75f4b4 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:16.788094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e82b6-7ee8-484a-95d1-20d78f959092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:16.789084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e82b6-7ee8-484a-95d1-20d78f959092 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.789604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e82b6-7ee8-484a-95d1-20d78f959092 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.789881 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e82b6-7ee8-484a-95d1-20d78f959092 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.790569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e82b6-7ee8-484a-95d1-20d78f959092 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:16.790874 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e82b6-7ee8-484a-95d1-20d78f959092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:16.792272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e82b6-7ee8-484a-95d1-20d78f959092 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:16.801621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 102/195] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:16.805723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cd2ac-a7f2-49c4-836a-a044a4b33c1d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:16.807413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e82b6-7ee8-484a-95d1-20d78f959092 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:16.844089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd319e-af42-4859-8de5-2aae75f07066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:16.844723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd319e-af42-4859-8de5-2aae75f07066 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.845005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd319e-af42-4859-8de5-2aae75f07066 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.845281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd319e-af42-4859-8de5-2aae75f07066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.896840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:16.920550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:16.925121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd319e-af42-4859-8de5-2aae75f07066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:16.926639 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd319e-af42-4859-8de5-2aae75f07066 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:16.929187 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e82b6-7ee8-484a-95d1-20d78f959092 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:16.970015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a8fed-7925-44ab-bfea-aae3ffe747a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:16.970656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8fed-7925-44ab-bfea-aae3ffe747a2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:16.970939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8fed-7925-44ab-bfea-aae3ffe747a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:16.971209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8fed-7925-44ab-bfea-aae3ffe747a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:16.987877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a8fed-7925-44ab-bfea-aae3ffe747a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-dd9a8fed-7925-44ab-bfea-aae3ffe747a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:16.994059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:16.994936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:16.996344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 103/196] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:16.996995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a8fed-7925-44ab-bfea-aae3ffe747a2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:17.057782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 94/197] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:17.058640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd319e-af42-4859-8de5-2aae75f07066 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:17.113724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a8fed-7925-44ab-bfea-aae3ffe747a2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:17.145996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:17.159542 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55047f4-c700-4192-b07b-69bce8118620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:17.160623 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55047f4-c700-4192-b07b-69bce8118620 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:17.161048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55047f4-c700-4192-b07b-69bce8118620 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:17.161438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55047f4-c700-4192-b07b-69bce8118620 None admin] PATH_INFO: `/v3/users/249e370f154d4748a72764c0aa73580e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:17.162784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55047f4-c700-4192-b07b-69bce8118620 None admin] RBAC: Authorizing `identity:get_user(user_id=249e370f154d4748a72764c0aa73580e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:17.164876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55047f4-c700-4192-b07b-69bce8118620 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:17.168153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 104/198] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:16 2022] GET /identity/v3/users/249e370f154d4748a72764c0aa73580e => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:17.168678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55047f4-c700-4192-b07b-69bce8118620 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:17.172754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd319e-af42-4859-8de5-2aae75f07066 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:17.197694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:17.215317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46663187-d028-4415-8693-0791ec98a4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:17.215941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46663187-d028-4415-8693-0791ec98a4f0 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:17.216222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46663187-d028-4415-8693-0791ec98a4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:17.216495 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46663187-d028-4415-8693-0791ec98a4f0 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:17.217297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46663187-d028-4415-8693-0791ec98a4f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:17.217634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46663187-d028-4415-8693-0791ec98a4f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:17.219072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46663187-d028-4415-8693-0791ec98a4f0 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:17.229041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 95/199] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:17.229993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46663187-d028-4415-8693-0791ec98a4f0 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:17.283640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55047f4-c700-4192-b07b-69bce8118620 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:17.323233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd87ef-c91f-4ac6-a161-a2fab56b41a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:17.324342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd87ef-c91f-4ac6-a161-a2fab56b41a0 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:17.324617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd87ef-c91f-4ac6-a161-a2fab56b41a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:17.324877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd87ef-c91f-4ac6-a161-a2fab56b41a0 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:17.326057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd87ef-c91f-4ac6-a161-a2fab56b41a0 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:17.327508 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd87ef-c91f-4ac6-a161-a2fab56b41a0 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:17.329116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 105/200] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:17 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:17.329890 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd87ef-c91f-4ac6-a161-a2fab56b41a0 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:17.367854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46663187-d028-4415-8693-0791ec98a4f0 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:17.409350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779eca5-52cd-485c-bc56-b99efb9ba9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:17.410003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779eca5-52cd-485c-bc56-b99efb9ba9e3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:17.410452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779eca5-52cd-485c-bc56-b99efb9ba9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:17.410734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779eca5-52cd-485c-bc56-b99efb9ba9e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:17.421319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:17.427864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:17.441480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd87ef-c91f-4ac6-a161-a2fab56b41a0 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:17.475756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f3820-b9be-43fc-b8c7-94e5ee7ca666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:17.476511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f3820-b9be-43fc-b8c7-94e5ee7ca666 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:17.476853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f3820-b9be-43fc-b8c7-94e5ee7ca666 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:17.477197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f3820-b9be-43fc-b8c7-94e5ee7ca666 None admin] PATH_INFO: `/v3/users/780865e68bbc4cc49315105a3c0b90f7` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:17.478411 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f3820-b9be-43fc-b8c7-94e5ee7ca666 None admin] RBAC: Authorizing `identity:get_user(user_id=780865e68bbc4cc49315105a3c0b90f7)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:17.480004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f3820-b9be-43fc-b8c7-94e5ee7ca666 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:17.481725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 106/201] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:17 2022] GET /identity/v3/users/780865e68bbc4cc49315105a3c0b90f7 => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:17.482605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f3820-b9be-43fc-b8c7-94e5ee7ca666 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:17.490475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779eca5-52cd-485c-bc56-b99efb9ba9e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:17.492339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779eca5-52cd-485c-bc56-b99efb9ba9e3 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:17.589375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f3820-b9be-43fc-b8c7-94e5ee7ca666 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:17.612036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 96/202] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:17.613061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779eca5-52cd-485c-bc56-b99efb9ba9e3 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:17.627789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5b3f3-aedc-4d94-aa62-55af8cce1229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:17.628515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5b3f3-aedc-4d94-aa62-55af8cce1229 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:17.628940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5b3f3-aedc-4d94-aa62-55af8cce1229 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:17.629319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5b3f3-aedc-4d94-aa62-55af8cce1229 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:17.630300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5b3f3-aedc-4d94-aa62-55af8cce1229 None admin] RBAC: Adding query filter params (role.id=8cd21d388193490bac3aa386bcc18763, scope.project.id=805a9d6e523c465393a8f47773ab2b23, user.id=249e370f154d4748a72764c0aa73580e) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:17.630690 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5b3f3-aedc-4d94-aa62-55af8cce1229 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:17.632284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5b3f3-aedc-4d94-aa62-55af8cce1229 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:17.638650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 107/203] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:17 2022] GET /identity/v3/role_assignments?user.id=249e370f154d4748a72764c0aa73580e&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=8cd21d388193490bac3aa386bcc18763 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:17.639692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5b3f3-aedc-4d94-aa62-55af8cce1229 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:17.670385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:17.684096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:17.817562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5b3f3-aedc-4d94-aa62-55af8cce1229 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:17.851089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e351ad1-42b3-40a3-a929-2b51d0df5b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:17.851651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e351ad1-42b3-40a3-a929-2b51d0df5b6d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:17.851926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e351ad1-42b3-40a3-a929-2b51d0df5b6d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:17.852192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e351ad1-42b3-40a3-a929-2b51d0df5b6d None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:17.861518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e351ad1-42b3-40a3-a929-2b51d0df5b6d None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:17.861518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e351ad1-42b3-40a3-a929-2b51d0df5b6d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:17.861518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 108/204] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:17 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:17.869522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e351ad1-42b3-40a3-a929-2b51d0df5b6d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:17.871741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779eca5-52cd-485c-bc56-b99efb9ba9e3 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:17.971605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:17.982740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e351ad1-42b3-40a3-a929-2b51d0df5b6d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:18.000318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1701d7-5001-47cb-9dc9-dd769b3ccd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:18.000318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1701d7-5001-47cb-9dc9-dd769b3ccd63 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:18.000318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1701d7-5001-47cb-9dc9-dd769b3ccd63 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:18.000318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1701d7-5001-47cb-9dc9-dd769b3ccd63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:18.058259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04f277-c7d7-40bd-8d00-20023dcce4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:18.058887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04f277-c7d7-40bd-8d00-20023dcce4fd None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:18.059162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04f277-c7d7-40bd-8d00-20023dcce4fd None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:18.060362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04f277-c7d7-40bd-8d00-20023dcce4fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:18.061112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04f277-c7d7-40bd-8d00-20023dcce4fd None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=805a9d6e523c465393a8f47773ab2b23, user.id=249e370f154d4748a72764c0aa73580e) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:18.064371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04f277-c7d7-40bd-8d00-20023dcce4fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:18.068117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04f277-c7d7-40bd-8d00-20023dcce4fd None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:18.097600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 109/205] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:17 2022] GET /identity/v3/role_assignments?user.id=249e370f154d4748a72764c0aa73580e&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:18.215834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1701d7-5001-47cb-9dc9-dd769b3ccd63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:18.217330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1701d7-5001-47cb-9dc9-dd769b3ccd63 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:18.304837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:18.403977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 97/206] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 791 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:18.409621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04f277-c7d7-40bd-8d00-20023dcce4fd None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:18.511805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04f277-c7d7-40bd-8d00-20023dcce4fd None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:18.550245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64b1a9-6949-48c7-ae84-195a4e318e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:18.550884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64b1a9-6949-48c7-ae84-195a4e318e56 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:18.551160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64b1a9-6949-48c7-ae84-195a4e318e56 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:18.551427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64b1a9-6949-48c7-ae84-195a4e318e56 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:18.552593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64b1a9-6949-48c7-ae84-195a4e318e56 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:18.554099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64b1a9-6949-48c7-ae84-195a4e318e56 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:18.555738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 110/207] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:18 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:18.561881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1701d7-5001-47cb-9dc9-dd769b3ccd63 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:18.688246 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1701d7-5001-47cb-9dc9-dd769b3ccd63 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:18.735599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab53e2-7310-4f23-b970-857266add92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:18.736219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab53e2-7310-4f23-b970-857266add92f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:18.736498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab53e2-7310-4f23-b970-857266add92f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:18.736767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab53e2-7310-4f23-b970-857266add92f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:18.737526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab53e2-7310-4f23-b970-857266add92f None admin] RBAC: Adding query filter params (role.id=8d46737ab3ff453a800fe6b47805eb5a, scope.project.id=130f64a6280e4b83bf885edf8986e895, user.id=780865e68bbc4cc49315105a3c0b90f7) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:18.737845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab53e2-7310-4f23-b970-857266add92f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:18.739288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab53e2-7310-4f23-b970-857266add92f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:18.750122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 98/208] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:18 2022] GET /identity/v3/role_assignments?user.id=780865e68bbc4cc49315105a3c0b90f7&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=8d46737ab3ff453a800fe6b47805eb5a => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:19.340109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c7a4a-9a20-46d3-8d76-b6aa6a8c5e39 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:19.340748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c7a4a-9a20-46d3-8d76-b6aa6a8c5e39 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:19.341243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c7a4a-9a20-46d3-8d76-b6aa6a8c5e39 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:19.343095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 111/209] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:19.348973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde895-711f-4425-83f0-d47c08d3badf None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:19.349811 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde895-711f-4425-83f0-d47c08d3badf None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:19.350295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde895-711f-4425-83f0-d47c08d3badf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:19.357328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:19.371789 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a5dde895-711f-4425-83f0-d47c08d3badf None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:19.373637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a5dde895-711f-4425-83f0-d47c08d3badf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:19.446054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 99/210] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:19.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef833098-bc97-4877-8f1a-c08911d9e5f0 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:19.526978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef833098-bc97-4877-8f1a-c08911d9e5f0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:19.527448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef833098-bc97-4877-8f1a-c08911d9e5f0 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:19.529022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 112/211] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:19.534951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a346-643c-41f8-843a-d1b2dda6123e None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:19.535462 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a346-643c-41f8-843a-d1b2dda6123e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:19.535928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a346-643c-41f8-843a-d1b2dda6123e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:19.554856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b158a346-643c-41f8-843a-d1b2dda6123e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:19.556303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b158a346-643c-41f8-843a-d1b2dda6123e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:19.628120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 100/212] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:19.739083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef833098-bc97-4877-8f1a-c08911d9e5f0 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:19.765255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:19.842270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef833098-bc97-4877-8f1a-c08911d9e5f0 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:19.876267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c98c58-fe44-4f65-b99a-b6c9ccd64171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:19.876891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c98c58-fe44-4f65-b99a-b6c9ccd64171 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:19.877162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c98c58-fe44-4f65-b99a-b6c9ccd64171 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:19.877428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c98c58-fe44-4f65-b99a-b6c9ccd64171 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:19.887387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c98c58-fe44-4f65-b99a-b6c9ccd64171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-88c98c58-fe44-4f65-b99a-b6c9ccd64171 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:19.892132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:19.892987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:19.893825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 113/213] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:19.899320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c98c58-fe44-4f65-b99a-b6c9ccd64171 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:19.899473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158a346-643c-41f8-843a-d1b2dda6123e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:19.967270 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:19.998411 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:20.016870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158a346-643c-41f8-843a-d1b2dda6123e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:20.027089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c98c58-fe44-4f65-b99a-b6c9ccd64171 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:20.062377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c1f8a-ab45-47f0-85a6-8c000099734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:20.063078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c1f8a-ab45-47f0-85a6-8c000099734b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.063359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c1f8a-ab45-47f0-85a6-8c000099734b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.063625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c1f8a-ab45-47f0-85a6-8c000099734b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.065406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c1f8a-ab45-47f0-85a6-8c000099734b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:20.067143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c1f8a-ab45-47f0-85a6-8c000099734b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:20.069420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 101/214] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:20.075261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c1f8a-ab45-47f0-85a6-8c000099734b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:20.076908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cb2f9-dcc5-4bb5-aa73-19dd63bcd2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:20.077885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cb2f9-dcc5-4bb5-aa73-19dd63bcd2b1 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.078396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cb2f9-dcc5-4bb5-aa73-19dd63bcd2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.078875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cb2f9-dcc5-4bb5-aa73-19dd63bcd2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.079778 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cb2f9-dcc5-4bb5-aa73-19dd63bcd2b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:20.080279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cb2f9-dcc5-4bb5-aa73-19dd63bcd2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:20.081883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cb2f9-dcc5-4bb5-aa73-19dd63bcd2b1 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:20.091607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 114/215] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:20.098452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cb2f9-dcc5-4bb5-aa73-19dd63bcd2b1 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:20.186700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c1f8a-ab45-47f0-85a6-8c000099734b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:20.206184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cb2f9-dcc5-4bb5-aa73-19dd63bcd2b1 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:20.218432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:20.224136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5aa2f-1d58-4f6a-ab6a-75648350feeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:20.224805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5aa2f-1d58-4f6a-ab6a-75648350feeb None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.225093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5aa2f-1d58-4f6a-ab6a-75648350feeb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.225368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5aa2f-1d58-4f6a-ab6a-75648350feeb None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.226062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5aa2f-1d58-4f6a-ab6a-75648350feeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:20.227513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5aa2f-1d58-4f6a-ab6a-75648350feeb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:20.244814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517965b-cd73-4ff3-bce3-54ec0dadbc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:20.245520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517965b-cd73-4ff3-bce3-54ec0dadbc99 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.245767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517965b-cd73-4ff3-bce3-54ec0dadbc99 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.246033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517965b-cd73-4ff3-bce3-54ec0dadbc99 None admin] PATH_INFO: `/v3/users/4612079bfff84228bf010c4f0990cfcf` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.249932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 102/216] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:38:20 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:38:20.250993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283c7d5-df27-4ab4-9cbb-c8d66ff4248a None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.251282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283c7d5-df27-4ab4-9cbb-c8d66ff4248a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.251559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283c7d5-df27-4ab4-9cbb-c8d66ff4248a None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.253322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 103/217] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:20.258887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b913b5a-cc5f-4896-8569-83d510f2a28f None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.259513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b913b5a-cc5f-4896-8569-83d510f2a28f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.260075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b913b5a-cc5f-4896-8569-83d510f2a28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.270004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517965b-cd73-4ff3-bce3-54ec0dadbc99 None admin] RBAC: Authorizing `identity:get_user(user_id=4612079bfff84228bf010c4f0990cfcf)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:20.271512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517965b-cd73-4ff3-bce3-54ec0dadbc99 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:20.273558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 115/218] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:20 2022] GET /identity/v3/users/4612079bfff84228bf010c4f0990cfcf => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:20.279469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517965b-cd73-4ff3-bce3-54ec0dadbc99 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:20.282490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:20.287990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-0b913b5a-cc5f-4896-8569-83d510f2a28f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:20.290194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0b913b5a-cc5f-4896-8569-83d510f2a28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:20.396681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 104/219] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:20.397382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517965b-cd73-4ff3-bce3-54ec0dadbc99 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:20.438747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980581d2-eb04-4e27-a730-6c63ed207f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:20.439357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980581d2-eb04-4e27-a730-6c63ed207f7f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.439631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980581d2-eb04-4e27-a730-6c63ed207f7f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.439905 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980581d2-eb04-4e27-a730-6c63ed207f7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.512520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980581d2-eb04-4e27-a730-6c63ed207f7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:20.514012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980581d2-eb04-4e27-a730-6c63ed207f7f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:20.515860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:20.623428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 116/220] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:20.628972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b913b5a-cc5f-4896-8569-83d510f2a28f None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:20.677042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556d5dc-1ddb-4290-be27-4214a97d2851 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.677372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556d5dc-1ddb-4290-be27-4214a97d2851 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.677609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556d5dc-1ddb-4290-be27-4214a97d2851 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.679212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 117/221] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:20.683324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556d5dc-1ddb-4290-be27-4214a97d2851 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:20.750571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b913b5a-cc5f-4896-8569-83d510f2a28f None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:20.768158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:20.788147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c220573-10d5-4bd0-9f83-7becf708a561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:20.788779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c220573-10d5-4bd0-9f83-7becf708a561 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.789056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c220573-10d5-4bd0-9f83-7becf708a561 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.789324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c220573-10d5-4bd0-9f83-7becf708a561 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.790888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c220573-10d5-4bd0-9f83-7becf708a561 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:20.793024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c220573-10d5-4bd0-9f83-7becf708a561 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:20.794715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 105/222] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:20 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:20.795889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae93c5b-62a8-46cb-bfaa-518c130d3c32 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.796171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae93c5b-62a8-46cb-bfaa-518c130d3c32 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.796443 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae93c5b-62a8-46cb-bfaa-518c130d3c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.819893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556d5dc-1ddb-4290-be27-4214a97d2851 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:20.821486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:20.822051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-dae93c5b-62a8-46cb-bfaa-518c130d3c32 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:20.824118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-dae93c5b-62a8-46cb-bfaa-518c130d3c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:20.858979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43129aec-5946-486b-9a76-52db05905739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:20.859616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43129aec-5946-486b-9a76-52db05905739 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:20.859890 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43129aec-5946-486b-9a76-52db05905739 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:20.860157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43129aec-5946-486b-9a76-52db05905739 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:20.872019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43129aec-5946-486b-9a76-52db05905739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-43129aec-5946-486b-9a76-52db05905739 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:20.877339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:20.878234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:20.879012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 118/223] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:20.880093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43129aec-5946-486b-9a76-52db05905739 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:20.905937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 106/224] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:20.906792 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae93c5b-62a8-46cb-bfaa-518c130d3c32 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:21.031087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43129aec-5946-486b-9a76-52db05905739 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:21.062606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae93c5b-62a8-46cb-bfaa-518c130d3c32 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:21.080743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:21.103732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5513a58-ccb2-4448-8573-b5bb41993194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:21.104769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5513a58-ccb2-4448-8573-b5bb41993194 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:21.105529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5513a58-ccb2-4448-8573-b5bb41993194 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:21.105946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5513a58-ccb2-4448-8573-b5bb41993194 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:21.106420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d217fa2-dae8-415e-a854-57702cfbf08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:21.106926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5513a58-ccb2-4448-8573-b5bb41993194 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:21.107225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d217fa2-dae8-415e-a854-57702cfbf08d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:21.107601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d217fa2-dae8-415e-a854-57702cfbf08d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:21.107706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5513a58-ccb2-4448-8573-b5bb41993194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:21.108035 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d217fa2-dae8-415e-a854-57702cfbf08d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:21.108866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d217fa2-dae8-415e-a854-57702cfbf08d None admin] RBAC: Adding query filter params (role.id=47112770ffdc45638da090c45073888e, scope.project.id=805a9d6e523c465393a8f47773ab2b23, user.id=4612079bfff84228bf010c4f0990cfcf) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:21.109285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d217fa2-dae8-415e-a854-57702cfbf08d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:21.109803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5513a58-ccb2-4448-8573-b5bb41993194 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:21.110927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d217fa2-dae8-415e-a854-57702cfbf08d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:21.119199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 107/225] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:21.124417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 119/226] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:20 2022] GET /identity/v3/role_assignments?user.id=4612079bfff84228bf010c4f0990cfcf&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=47112770ffdc45638da090c45073888e => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:21.130465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5513a58-ccb2-4448-8573-b5bb41993194 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:21.201612 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d217fa2-dae8-415e-a854-57702cfbf08d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:21.232478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:21.255684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5513a58-ccb2-4448-8573-b5bb41993194 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:21.308506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3f4d1-0ebe-474e-a2f7-b758c1739d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:21.308506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3f4d1-0ebe-474e-a2f7-b758c1739d5b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:21.308506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3f4d1-0ebe-474e-a2f7-b758c1739d5b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:21.308506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3f4d1-0ebe-474e-a2f7-b758c1739d5b None admin] PATH_INFO: `/v3/users/249e370f154d4748a72764c0aa73580e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:21.310165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3f4d1-0ebe-474e-a2f7-b758c1739d5b None admin] RBAC: Authorizing `identity:get_user(user_id=249e370f154d4748a72764c0aa73580e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:21.311974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3f4d1-0ebe-474e-a2f7-b758c1739d5b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:21.318920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 108/227] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:21 2022] GET /identity/v3/users/249e370f154d4748a72764c0aa73580e => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:21.320701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3f4d1-0ebe-474e-a2f7-b758c1739d5b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:21.360875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d217fa2-dae8-415e-a854-57702cfbf08d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:21.417845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:21.433957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc370a-a6ed-4cd5-b2f5-35392e71db6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:21.434597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc370a-a6ed-4cd5-b2f5-35392e71db6c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:21.434877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc370a-a6ed-4cd5-b2f5-35392e71db6c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:21.435149 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc370a-a6ed-4cd5-b2f5-35392e71db6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:21.461758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc370a-a6ed-4cd5-b2f5-35392e71db6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-f5bc370a-a6ed-4cd5-b2f5-35392e71db6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:21.468392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:21.469266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:21.470084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:21.473126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 120/228] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:38:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:21.477123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc370a-a6ed-4cd5-b2f5-35392e71db6c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:21.507612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3f4d1-0ebe-474e-a2f7-b758c1739d5b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:21.601345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:21.618840 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc370a-a6ed-4cd5-b2f5-35392e71db6c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:21.630202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490f892-1a1f-43f4-a620-aab83e66b083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:21.631009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490f892-1a1f-43f4-a620-aab83e66b083 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:21.631299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490f892-1a1f-43f4-a620-aab83e66b083 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:21.631580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490f892-1a1f-43f4-a620-aab83e66b083 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:21.632746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490f892-1a1f-43f4-a620-aab83e66b083 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:21.634253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490f892-1a1f-43f4-a620-aab83e66b083 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:21.636230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 109/229] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:21 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:21.641655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490f892-1a1f-43f4-a620-aab83e66b083 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:21.662865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc464d44-7978-4b8e-b81b-778e47017cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:21.663468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc464d44-7978-4b8e-b81b-778e47017cc8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:21.663746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc464d44-7978-4b8e-b81b-778e47017cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:21.665487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc464d44-7978-4b8e-b81b-778e47017cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:21.666818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc464d44-7978-4b8e-b81b-778e47017cc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:21.667652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc464d44-7978-4b8e-b81b-778e47017cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:21.671115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc464d44-7978-4b8e-b81b-778e47017cc8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:21.689469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 121/230] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:21.695417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc464d44-7978-4b8e-b81b-778e47017cc8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:21.764349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490f892-1a1f-43f4-a620-aab83e66b083 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:21.793903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:21.809323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65f8c6-0fcd-4b86-afef-50c12d8c088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:21.810035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65f8c6-0fcd-4b86-afef-50c12d8c088f None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:21.810315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65f8c6-0fcd-4b86-afef-50c12d8c088f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:21.810589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65f8c6-0fcd-4b86-afef-50c12d8c088f None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23/users/249e370f154d4748a72764c0aa73580e/roles/8cd21d388193490bac3aa386bcc18763` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:21.812844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65f8c6-0fcd-4b86-afef-50c12d8c088f None admin] RBAC: Authorizing `identity:create_grant(project_id=805a9d6e523c465393a8f47773ab2b23, user_id=249e370f154d4748a72764c0aa73580e, role_id=8cd21d388193490bac3aa386bcc18763)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:21.814375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65f8c6-0fcd-4b86-afef-50c12d8c088f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:21.830891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 110/231] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:38:21 2022] PUT /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23/users/249e370f154d4748a72764c0aa73580e/roles/8cd21d388193490bac3aa386bcc18763 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:21.897665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc464d44-7978-4b8e-b81b-778e47017cc8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:21.957052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9076f-d338-4a36-9dff-63e9ade18293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:21.957747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9076f-d338-4a36-9dff-63e9ade18293 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:21.958020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9076f-d338-4a36-9dff-63e9ade18293 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:21.958289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9076f-d338-4a36-9dff-63e9ade18293 None admin] PATH_INFO: `/v3/users/249e370f154d4748a72764c0aa73580e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:21.959538 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9076f-d338-4a36-9dff-63e9ade18293 None admin] RBAC: Authorizing `identity:get_user(user_id=249e370f154d4748a72764c0aa73580e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:21.961008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9076f-d338-4a36-9dff-63e9ade18293 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:21.962667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 122/232] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:21 2022] GET /identity/v3/users/249e370f154d4748a72764c0aa73580e => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:21.968611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65f8c6-0fcd-4b86-afef-50c12d8c088f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:22.105960 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65f8c6-0fcd-4b86-afef-50c12d8c088f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:22.143154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b0651-728b-4527-9926-781963ca83a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:22.143853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b0651-728b-4527-9926-781963ca83a5 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:22.144137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b0651-728b-4527-9926-781963ca83a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:22.144412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b0651-728b-4527-9926-781963ca83a5 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:22.145590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b0651-728b-4527-9926-781963ca83a5 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:22.147041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b0651-728b-4527-9926-781963ca83a5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:22.149140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 111/233] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:21 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:22.154937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9076f-d338-4a36-9dff-63e9ade18293 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:22.184710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:22.278552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9076f-d338-4a36-9dff-63e9ade18293 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:22.320478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627c4ab-395d-47d9-9815-e22d9b924c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:22.321146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627c4ab-395d-47d9-9815-e22d9b924c89 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:22.321420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627c4ab-395d-47d9-9815-e22d9b924c89 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:22.321728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627c4ab-395d-47d9-9815-e22d9b924c89 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23/users/249e370f154d4748a72764c0aa73580e/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:22.326230 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627c4ab-395d-47d9-9815-e22d9b924c89 None admin] RBAC: Authorizing `identity:create_grant(project_id=805a9d6e523c465393a8f47773ab2b23, user_id=249e370f154d4748a72764c0aa73580e, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:22.327683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627c4ab-395d-47d9-9815-e22d9b924c89 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:22.348207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 123/234] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:38:22 2022] PUT /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23/users/249e370f154d4748a72764c0aa73580e/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:22.929858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b1a4-fe04-4588-b330-36fd456ab02e None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:22.930417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b1a4-fe04-4588-b330-36fd456ab02e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:22.930417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b1a4-fe04-4588-b330-36fd456ab02e None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:22.931962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 112/235] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:22.937194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f954e-d82b-40fd-b6b0-70ef77519ff2 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:22.937618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f954e-d82b-40fd-b6b0-70ef77519ff2 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:22.937817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f954e-d82b-40fd-b6b0-70ef77519ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:22.960671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-257f954e-d82b-40fd-b6b0-70ef77519ff2 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:22.961928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-257f954e-d82b-40fd-b6b0-70ef77519ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:23.059179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 124/236] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:23.119757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941867-c076-4a44-a02a-5b6daa916a64 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:23.120161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941867-c076-4a44-a02a-5b6daa916a64 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:23.120443 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941867-c076-4a44-a02a-5b6daa916a64 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:23.121986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 113/237] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:23.127303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2dc37-ca60-4005-ad91-b1f0d65f3449 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:23.127588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2dc37-ca60-4005-ad91-b1f0d65f3449 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:23.127861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2dc37-ca60-4005-ad91-b1f0d65f3449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:23.144562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:23.154813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-fbd2dc37-ca60-4005-ad91-b1f0d65f3449 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:23.156009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-fbd2dc37-ca60-4005-ad91-b1f0d65f3449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:23.245252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 125/238] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:23.436748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51941867-c076-4a44-a02a-5b6daa916a64 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:23.493598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:23.570185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51941867-c076-4a44-a02a-5b6daa916a64 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:23.609389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c61ca2-160f-412c-a681-fc101bb5d409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:23.610085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c61ca2-160f-412c-a681-fc101bb5d409 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:23.610368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c61ca2-160f-412c-a681-fc101bb5d409 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:23.610644 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c61ca2-160f-412c-a681-fc101bb5d409 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:23.611812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c61ca2-160f-412c-a681-fc101bb5d409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:23.613823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c61ca2-160f-412c-a681-fc101bb5d409 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:23.616448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 114/239] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:23.622461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2dc37-ca60-4005-ad91-b1f0d65f3449 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:23.688073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c61ca2-160f-412c-a681-fc101bb5d409 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:23.781833 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2dc37-ca60-4005-ad91-b1f0d65f3449 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:23.817714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c61ca2-160f-412c-a681-fc101bb5d409 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:23.819981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecd4e9-f9ca-44bd-948e-71d97202d2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:23.821027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecd4e9-f9ca-44bd-948e-71d97202d2a0 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:23.821523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecd4e9-f9ca-44bd-948e-71d97202d2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:23.822007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecd4e9-f9ca-44bd-948e-71d97202d2a0 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:23.822841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecd4e9-f9ca-44bd-948e-71d97202d2a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:23.825007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecd4e9-f9ca-44bd-948e-71d97202d2a0 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:23.827688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:23.837866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:23.860066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 126/240] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:38:23 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:38:23.861790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945db544-2b6d-4c1b-959e-cf08d2af5e70 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:23.862338 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945db544-2b6d-4c1b-959e-cf08d2af5e70 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:23.862848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945db544-2b6d-4c1b-959e-cf08d2af5e70 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:23.864462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 127/241] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:23.870414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd50edd-8d77-4e10-b152-28720316a7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:23.871469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd50edd-8d77-4e10-b152-28720316a7b8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:23.871951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd50edd-8d77-4e10-b152-28720316a7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:23.872413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd50edd-8d77-4e10-b152-28720316a7b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:23.875258 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19689e-3bfe-4953-b226-f344d910fa53 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:23.875806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19689e-3bfe-4953-b226-f344d910fa53 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:23.876264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19689e-3bfe-4953-b226-f344d910fa53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:23.905777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd50edd-8d77-4e10-b152-28720316a7b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-ddd50edd-8d77-4e10-b152-28720316a7b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:23.916355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:23.917369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:23.918203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:23.922397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5a19689e-3bfe-4953-b226-f344d910fa53 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:23.923923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5a19689e-3bfe-4953-b226-f344d910fa53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:23.924618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 115/242] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:23.935527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd50edd-8d77-4e10-b152-28720316a7b8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:24.068267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 128/243] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:24.095795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd50edd-8d77-4e10-b152-28720316a7b8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:24.134080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c7358-dab0-4a6c-a8bd-a9dbe13b41ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:24.134701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c7358-dab0-4a6c-a8bd-a9dbe13b41ad None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:24.134982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c7358-dab0-4a6c-a8bd-a9dbe13b41ad None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:24.135339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c7358-dab0-4a6c-a8bd-a9dbe13b41ad None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:24.136636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c7358-dab0-4a6c-a8bd-a9dbe13b41ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:24.136956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c7358-dab0-4a6c-a8bd-a9dbe13b41ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:24.138409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c7358-dab0-4a6c-a8bd-a9dbe13b41ad None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:24.147216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 116/244] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:24.152988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19689e-3bfe-4953-b226-f344d910fa53 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:24.258549 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19689e-3bfe-4953-b226-f344d910fa53 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:24.289164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:24.306886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de231f-5dd4-4143-8d7d-c6aaa319ca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:24.308084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de231f-5dd4-4143-8d7d-c6aaa319ca2c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:24.308370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de231f-5dd4-4143-8d7d-c6aaa319ca2c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:24.308650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de231f-5dd4-4143-8d7d-c6aaa319ca2c None admin] PATH_INFO: `/v3/users/f8cd5b61edb44dcdaf63f403b2c88a42` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:24.323104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de231f-5dd4-4143-8d7d-c6aaa319ca2c None admin] RBAC: Authorizing `identity:get_user(user_id=f8cd5b61edb44dcdaf63f403b2c88a42)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:24.324741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de231f-5dd4-4143-8d7d-c6aaa319ca2c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:24.326676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 129/245] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:24 2022] GET /identity/v3/users/f8cd5b61edb44dcdaf63f403b2c88a42 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:24.331963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c7358-dab0-4a6c-a8bd-a9dbe13b41ad None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:24.364588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:24.380631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de231f-5dd4-4143-8d7d-c6aaa319ca2c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:24.454478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c7358-dab0-4a6c-a8bd-a9dbe13b41ad None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:24.498135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e7063-72cd-4278-b07e-3d7ce26beade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:24.498804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e7063-72cd-4278-b07e-3d7ce26beade None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:24.499137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e7063-72cd-4278-b07e-3d7ce26beade None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:24.499453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e7063-72cd-4278-b07e-3d7ce26beade None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:24.505957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de231f-5dd4-4143-8d7d-c6aaa319ca2c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:24.548836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5a4de-0e11-47e3-b316-e13e5e1fe51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:24.549625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5a4de-0e11-47e3-b316-e13e5e1fe51d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:24.549906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5a4de-0e11-47e3-b316-e13e5e1fe51d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:24.550179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5a4de-0e11-47e3-b316-e13e5e1fe51d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:24.562302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5a4de-0e11-47e3-b316-e13e5e1fe51d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-b8b5a4de-0e11-47e3-b316-e13e5e1fe51d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:24.567638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:24.568564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:24.569807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 130/246] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:24.577166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5a4de-0e11-47e3-b316-e13e5e1fe51d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:24.581317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e7063-72cd-4278-b07e-3d7ce26beade None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:24.582829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e7063-72cd-4278-b07e-3d7ce26beade None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:24.617115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:24.636990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:24.701645 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5a4de-0e11-47e3-b316-e13e5e1fe51d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:24.704246 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 117/247] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:24.715688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e7063-72cd-4278-b07e-3d7ce26beade None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:24.751272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90dd90-a9bc-433f-a7f2-f7e09eca4ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:24.752024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90dd90-a9bc-433f-a7f2-f7e09eca4ff6 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:24.752657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90dd90-a9bc-433f-a7f2-f7e09eca4ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:24.753073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90dd90-a9bc-433f-a7f2-f7e09eca4ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:24.753921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90dd90-a9bc-433f-a7f2-f7e09eca4ff6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:24.754227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90dd90-a9bc-433f-a7f2-f7e09eca4ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:24.755798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90dd90-a9bc-433f-a7f2-f7e09eca4ff6 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:24.769059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 131/248] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:24.770781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140439e2-cf82-45e5-a762-7cd9fd0d2b69 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:24.771059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140439e2-cf82-45e5-a762-7cd9fd0d2b69 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:24.771324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140439e2-cf82-45e5-a762-7cd9fd0d2b69 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:24.772666 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 132/249] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:24.777608 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140439e2-cf82-45e5-a762-7cd9fd0d2b69 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:24.835363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e7063-72cd-4278-b07e-3d7ce26beade None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:24.876882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:24.884607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c60bb-27d7-4cbd-8d95-bd9655a82617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:24.885473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c60bb-27d7-4cbd-8d95-bd9655a82617 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:24.886500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c60bb-27d7-4cbd-8d95-bd9655a82617 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:24.886908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c60bb-27d7-4cbd-8d95-bd9655a82617 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:24.888181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c60bb-27d7-4cbd-8d95-bd9655a82617 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:24.889838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c60bb-27d7-4cbd-8d95-bd9655a82617 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:24.891610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 118/250] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:24 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:24.893093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4efe52-2cf7-4bb3-821e-ee250ca6c3aa None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:24.893584 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4efe52-2cf7-4bb3-821e-ee250ca6c3aa None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:24.893991 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4efe52-2cf7-4bb3-821e-ee250ca6c3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:24.906453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140439e2-cf82-45e5-a762-7cd9fd0d2b69 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:24.908301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:24.913817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6d4efe52-2cf7-4bb3-821e-ee250ca6c3aa None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:24.915214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6d4efe52-2cf7-4bb3-821e-ee250ca6c3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:24.944655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc7e41-a4a4-4567-a21f-aa6d276116df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:24.945483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc7e41-a4a4-4567-a21f-aa6d276116df None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:24.945901 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc7e41-a4a4-4567-a21f-aa6d276116df None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:24.946238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc7e41-a4a4-4567-a21f-aa6d276116df None admin] PATH_INFO: `/v3/users/4612079bfff84228bf010c4f0990cfcf` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:24.947651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc7e41-a4a4-4567-a21f-aa6d276116df None admin] RBAC: Authorizing `identity:get_user(user_id=4612079bfff84228bf010c4f0990cfcf)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:24.950328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc7e41-a4a4-4567-a21f-aa6d276116df None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:24.952062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 133/251] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:24 2022] GET /identity/v3/users/4612079bfff84228bf010c4f0990cfcf => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:24.953014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc7e41-a4a4-4567-a21f-aa6d276116df None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:25.023765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 119/252] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:25.025308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4efe52-2cf7-4bb3-821e-ee250ca6c3aa None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:25.093189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc7e41-a4a4-4567-a21f-aa6d276116df None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:25.156102 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959433e-01f8-4866-8654-ae063d12f3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:25.156720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959433e-01f8-4866-8654-ae063d12f3eb None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:25.157362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959433e-01f8-4866-8654-ae063d12f3eb None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:25.157362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959433e-01f8-4866-8654-ae063d12f3eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:25.161522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959433e-01f8-4866-8654-ae063d12f3eb None admin] RBAC: Adding query filter params (role.id=8d46737ab3ff453a800fe6b47805eb5a, scope.project.id=805a9d6e523c465393a8f47773ab2b23, user.id=f8cd5b61edb44dcdaf63f403b2c88a42) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:25.161522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959433e-01f8-4866-8654-ae063d12f3eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:25.161522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959433e-01f8-4866-8654-ae063d12f3eb None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:25.177525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 134/253] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:24 2022] GET /identity/v3/role_assignments?user.id=f8cd5b61edb44dcdaf63f403b2c88a42&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=8d46737ab3ff453a800fe6b47805eb5a => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:25.201600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4efe52-2cf7-4bb3-821e-ee250ca6c3aa None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:25.244525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:25.255337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7caa1-a206-44c0-a998-9a57cde8fc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:25.256325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7caa1-a206-44c0-a998-9a57cde8fc78 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:25.256791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7caa1-a206-44c0-a998-9a57cde8fc78 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:25.257227 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7caa1-a206-44c0-a998-9a57cde8fc78 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:25.258602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7caa1-a206-44c0-a998-9a57cde8fc78 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:25.260259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7caa1-a206-44c0-a998-9a57cde8fc78 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:25.262208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 120/254] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:25 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:25.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959433e-01f8-4866-8654-ae063d12f3eb None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:25.345190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:25.434694 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7caa1-a206-44c0-a998-9a57cde8fc78 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:25.434883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959433e-01f8-4866-8654-ae063d12f3eb None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:25.478924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b70d9d-74f1-45d9-85d5-3ccf8502e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:25.479653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b70d9d-74f1-45d9-85d5-3ccf8502e8e4 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:25.479927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b70d9d-74f1-45d9-85d5-3ccf8502e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:25.480197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b70d9d-74f1-45d9-85d5-3ccf8502e8e4 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23/users/4612079bfff84228bf010c4f0990cfcf/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:25.486189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b70d9d-74f1-45d9-85d5-3ccf8502e8e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=805a9d6e523c465393a8f47773ab2b23, user_id=4612079bfff84228bf010c4f0990cfcf, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:25.487686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b70d9d-74f1-45d9-85d5-3ccf8502e8e4 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:25.512209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 135/255] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:38:25 2022] PUT /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23/users/4612079bfff84228bf010c4f0990cfcf/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:25.532772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53260884-a287-41b0-a7c9-068f15e0f0aa None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:25.533175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53260884-a287-41b0-a7c9-068f15e0f0aa None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:25.533519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53260884-a287-41b0-a7c9-068f15e0f0aa None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:25.535307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 136/256] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:25.543844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a093-1216-4cc7-9e6e-54fa802b7958 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:25.544133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a093-1216-4cc7-9e6e-54fa802b7958 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:25.544387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a093-1216-4cc7-9e6e-54fa802b7958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:25.573809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e1e2a093-1216-4cc7-9e6e-54fa802b7958 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:25.577545 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e1e2a093-1216-4cc7-9e6e-54fa802b7958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:25.652009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7caa1-a206-44c0-a998-9a57cde8fc78 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:25.690140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 137/257] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:25.709841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f64c9-98d5-4b7e-8c98-ee1a7a60b70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:25.710481 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f64c9-98d5-4b7e-8c98-ee1a7a60b70b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:25.710801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f64c9-98d5-4b7e-8c98-ee1a7a60b70b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:25.711074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f64c9-98d5-4b7e-8c98-ee1a7a60b70b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:25.727182 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f64c9-98d5-4b7e-8c98-ee1a7a60b70b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-ad5f64c9-98d5-4b7e-8c98-ee1a7a60b70b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:25.732320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:25.733173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:25.733969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 121/258] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:25.740948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2a093-1216-4cc7-9e6e-54fa802b7958 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:25.773537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:25.872275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2a093-1216-4cc7-9e6e-54fa802b7958 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:25.913772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b37d6c-b305-4af8-9f75-8f8e8e07e9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:25.914771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b37d6c-b305-4af8-9f75-8f8e8e07e9cd None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:25.915283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b37d6c-b305-4af8-9f75-8f8e8e07e9cd None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:25.915758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b37d6c-b305-4af8-9f75-8f8e8e07e9cd None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:25.916632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b37d6c-b305-4af8-9f75-8f8e8e07e9cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:25.917678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b37d6c-b305-4af8-9f75-8f8e8e07e9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:25.919302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b37d6c-b305-4af8-9f75-8f8e8e07e9cd None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:25.928775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 138/259] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:25.938040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f64c9-98d5-4b7e-8c98-ee1a7a60b70b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:26.014761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:26.059697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f64c9-98d5-4b7e-8c98-ee1a7a60b70b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:26.083603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b37d6c-b305-4af8-9f75-8f8e8e07e9cd None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:26.093758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58a854-2e6a-4bfe-9c7f-a23e89f46dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:26.094395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58a854-2e6a-4bfe-9c7f-a23e89f46dde None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:26.094675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58a854-2e6a-4bfe-9c7f-a23e89f46dde None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:26.094946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58a854-2e6a-4bfe-9c7f-a23e89f46dde None admin] PATH_INFO: `/v3/users/249e370f154d4748a72764c0aa73580e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:26.096126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58a854-2e6a-4bfe-9c7f-a23e89f46dde None admin] RBAC: Authorizing `identity:get_user(user_id=249e370f154d4748a72764c0aa73580e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:26.097753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58a854-2e6a-4bfe-9c7f-a23e89f46dde None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:26.099373 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 122/260] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:25 2022] GET /identity/v3/users/249e370f154d4748a72764c0aa73580e => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:26.105300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58a854-2e6a-4bfe-9c7f-a23e89f46dde None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:26.261132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58a854-2e6a-4bfe-9c7f-a23e89f46dde None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:26.293930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:26.295190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:26.300517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91360d-798e-41d7-a47c-3689dd21e1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:26.301199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91360d-798e-41d7-a47c-3689dd21e1b7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:26.301481 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91360d-798e-41d7-a47c-3689dd21e1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:26.301780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91360d-798e-41d7-a47c-3689dd21e1b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:26.320725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b37d6c-b305-4af8-9f75-8f8e8e07e9cd None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:26.388589 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66927c2f-c82e-49a8-8ff0-0364b7f70104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:26.389231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66927c2f-c82e-49a8-8ff0-0364b7f70104 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:26.389529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66927c2f-c82e-49a8-8ff0-0364b7f70104 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:26.389808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66927c2f-c82e-49a8-8ff0-0364b7f70104 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:26.411731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66927c2f-c82e-49a8-8ff0-0364b7f70104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-66927c2f-c82e-49a8-8ff0-0364b7f70104 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:26.419046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:26.420414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:26.421691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:26.426177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 139/261] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:26.430200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66927c2f-c82e-49a8-8ff0-0364b7f70104 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:26.430955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91360d-798e-41d7-a47c-3689dd21e1b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:26.432996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91360d-798e-41d7-a47c-3689dd21e1b7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:26.627646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 123/262] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 14:38:26.630589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91360d-798e-41d7-a47c-3689dd21e1b7 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:26.667484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:26.679409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66927c2f-c82e-49a8-8ff0-0364b7f70104 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:26.760962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91360d-798e-41d7-a47c-3689dd21e1b7 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:26.770192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f6c0f-59f5-4252-9650-17072e0ce018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:26.770814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f6c0f-59f5-4252-9650-17072e0ce018 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:26.771086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f6c0f-59f5-4252-9650-17072e0ce018 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:26.771348 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f6c0f-59f5-4252-9650-17072e0ce018 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:26.772021 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f6c0f-59f5-4252-9650-17072e0ce018 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:26.772321 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f6c0f-59f5-4252-9650-17072e0ce018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:26.773739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f6c0f-59f5-4252-9650-17072e0ce018 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:26.792630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 140/263] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:26.798080 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f6c0f-59f5-4252-9650-17072e0ce018 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:26.808392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2de032-abec-4241-965a-c4979a0545bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:26.809085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2de032-abec-4241-965a-c4979a0545bc None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:26.809370 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2de032-abec-4241-965a-c4979a0545bc None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:26.809714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2de032-abec-4241-965a-c4979a0545bc None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:26.810859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2de032-abec-4241-965a-c4979a0545bc None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:26.812315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2de032-abec-4241-965a-c4979a0545bc None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:26.814038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 124/264] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:26 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:26.819767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2de032-abec-4241-965a-c4979a0545bc None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:26.829016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:26.924740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2de032-abec-4241-965a-c4979a0545bc None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:26.932946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:27.007448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9f28e-5be7-46e3-9329-14c54cf923a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:27.008407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9f28e-5be7-46e3-9329-14c54cf923a9 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:27.008962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9f28e-5be7-46e3-9329-14c54cf923a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:27.010171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9f28e-5be7-46e3-9329-14c54cf923a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:27.011747 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9f28e-5be7-46e3-9329-14c54cf923a9 None admin] RBAC: Adding query filter params (role.id=8cd21d388193490bac3aa386bcc18763, scope.project.id=805a9d6e523c465393a8f47773ab2b23, user.id=249e370f154d4748a72764c0aa73580e) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:27.012405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9f28e-5be7-46e3-9329-14c54cf923a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:27.014826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9f28e-5be7-46e3-9329-14c54cf923a9 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:27.037005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 125/265] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:26 2022] GET /identity/v3/role_assignments?user.id=249e370f154d4748a72764c0aa73580e&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=8cd21d388193490bac3aa386bcc18763 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:27.051294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f6c0f-59f5-4252-9650-17072e0ce018 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:27.100906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49028c1b-2f3a-46dc-9cba-af89078215a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:27.101614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49028c1b-2f3a-46dc-9cba-af89078215a9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:27.101899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49028c1b-2f3a-46dc-9cba-af89078215a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:27.102168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49028c1b-2f3a-46dc-9cba-af89078215a9 None admin] PATH_INFO: `/v3/users/249e370f154d4748a72764c0aa73580e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:27.103310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49028c1b-2f3a-46dc-9cba-af89078215a9 None admin] RBAC: Authorizing `identity:get_user(user_id=249e370f154d4748a72764c0aa73580e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:27.104845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49028c1b-2f3a-46dc-9cba-af89078215a9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:27.106725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 141/266] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:26 2022] GET /identity/v3/users/249e370f154d4748a72764c0aa73580e => generated 283 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:27.112201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9f28e-5be7-46e3-9329-14c54cf923a9 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:27.261064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9f28e-5be7-46e3-9329-14c54cf923a9 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:27.315105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfcd8d-8505-4179-9bac-b8d5c5905ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:27.315761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfcd8d-8505-4179-9bac-b8d5c5905ed3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:27.316042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfcd8d-8505-4179-9bac-b8d5c5905ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:27.316312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfcd8d-8505-4179-9bac-b8d5c5905ed3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:27.350049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:27.409222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bfcd8d-8505-4179-9bac-b8d5c5905ed3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:27.410726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bfcd8d-8505-4179-9bac-b8d5c5905ed3 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:27.424333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b3608-711d-4098-a0ab-d370d1f1ff3b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:27.424844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b3608-711d-4098-a0ab-d370d1f1ff3b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:27.424947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b3608-711d-4098-a0ab-d370d1f1ff3b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:27.426447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 142/267] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:38:27.435980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397b2e7-2735-41d0-9f53-325ac1a16115 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:27.436296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397b2e7-2735-41d0-9f53-325ac1a16115 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:27.436534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397b2e7-2735-41d0-9f53-325ac1a16115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:27.471144 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f397b2e7-2735-41d0-9f53-325ac1a16115 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:27.472336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f397b2e7-2735-41d0-9f53-325ac1a16115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:27.538552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 126/268] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:27.546633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfcd8d-8505-4179-9bac-b8d5c5905ed3 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:27.605888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:27.641281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 143/269] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:27.654424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfcd8d-8505-4179-9bac-b8d5c5905ed3 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:27.689370 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8ea84-81e1-4361-8e29-976df2f2e5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:27.690038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8ea84-81e1-4361-8e29-976df2f2e5f4 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:27.690319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8ea84-81e1-4361-8e29-976df2f2e5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:27.690591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8ea84-81e1-4361-8e29-976df2f2e5f4 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:27.691728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8ea84-81e1-4361-8e29-976df2f2e5f4 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:27.693177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8ea84-81e1-4361-8e29-976df2f2e5f4 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:27.695012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 127/270] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:27 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:27.701168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397b2e7-2735-41d0-9f53-325ac1a16115 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:27.712625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:27.817421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397b2e7-2735-41d0-9f53-325ac1a16115 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:27.859023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e452e32-5880-46e4-a1dc-f5ccd59b4f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:27.859625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e452e32-5880-46e4-a1dc-f5ccd59b4f2c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:27.859897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e452e32-5880-46e4-a1dc-f5ccd59b4f2c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:27.860170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e452e32-5880-46e4-a1dc-f5ccd59b4f2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:27.860887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e452e32-5880-46e4-a1dc-f5ccd59b4f2c None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=805a9d6e523c465393a8f47773ab2b23, user.id=249e370f154d4748a72764c0aa73580e) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:27.861188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e452e32-5880-46e4-a1dc-f5ccd59b4f2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:27.862652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e452e32-5880-46e4-a1dc-f5ccd59b4f2c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:27.871146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 144/271] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:27 2022] GET /identity/v3/role_assignments?user.id=249e370f154d4748a72764c0aa73580e&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:28.030914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b833005-4ff7-445c-a25e-3a533a415e08 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.031292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b833005-4ff7-445c-a25e-3a533a415e08 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.031474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b833005-4ff7-445c-a25e-3a533a415e08 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.032851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 128/272] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:28.050283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d85954-35ad-44e8-8676-86fbc2c69926 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.050604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d85954-35ad-44e8-8676-86fbc2c69926 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.050817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d85954-35ad-44e8-8676-86fbc2c69926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.073251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-47d85954-35ad-44e8-8676-86fbc2c69926 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:28.074415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-47d85954-35ad-44e8-8676-86fbc2c69926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:28.121465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b833005-4ff7-445c-a25e-3a533a415e08 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:28.145960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:28.153028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 145/273] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:28.203181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0e695-5936-4d52-bc36-9f0691289622 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.203181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0e695-5936-4d52-bc36-9f0691289622 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.203445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0e695-5936-4d52-bc36-9f0691289622 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.204644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 146/274] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:38:28.210741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d90f6-aff5-4122-a51d-d72cd402cbcf None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.210741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d90f6-aff5-4122-a51d-d72cd402cbcf None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.210741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d90f6-aff5-4122-a51d-d72cd402cbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.228837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f23d90f6-aff5-4122-a51d-d72cd402cbcf None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:28.231174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f23d90f6-aff5-4122-a51d-d72cd402cbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:28.259641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b833005-4ff7-445c-a25e-3a533a415e08 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:28.289786 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:28.313137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a08cb-0a44-4db5-9230-1940a8dd116e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:28.314081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a08cb-0a44-4db5-9230-1940a8dd116e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.314500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a08cb-0a44-4db5-9230-1940a8dd116e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.314912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a08cb-0a44-4db5-9230-1940a8dd116e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.328715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a08cb-0a44-4db5-9230-1940a8dd116e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:28.331862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 147/275] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-fb4a08cb-0a44-4db5-9230-1940a8dd116e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:28.334633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:28.335450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:28.336250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:28.338450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 129/276] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:28.349583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d90f6-aff5-4122-a51d-d72cd402cbcf None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:28.456896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d90f6-aff5-4122-a51d-d72cd402cbcf None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:28.468078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a08cb-0a44-4db5-9230-1940a8dd116e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:28.496879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:28.505961 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d64954-8bb7-43e8-b389-33b9d00faf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:28.506765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d64954-8bb7-43e8-b389-33b9d00faf82 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.507949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d64954-8bb7-43e8-b389-33b9d00faf82 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.508222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d64954-8bb7-43e8-b389-33b9d00faf82 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.508898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d64954-8bb7-43e8-b389-33b9d00faf82 None admin] RBAC: Adding query filter params (name=member) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:28.509197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d64954-8bb7-43e8-b389-33b9d00faf82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:28.511195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d64954-8bb7-43e8-b389-33b9d00faf82 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:28.522309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 148/277] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:28.527821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d64954-8bb7-43e8-b389-33b9d00faf82 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:28.586551 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a08cb-0a44-4db5-9230-1940a8dd116e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:28.624268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394c9c1-f43c-44ef-94eb-a6c1b2acbc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:28.624998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394c9c1-f43c-44ef-94eb-a6c1b2acbc9f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.625372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394c9c1-f43c-44ef-94eb-a6c1b2acbc9f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.625742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394c9c1-f43c-44ef-94eb-a6c1b2acbc9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.627142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394c9c1-f43c-44ef-94eb-a6c1b2acbc9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:28.628538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394c9c1-f43c-44ef-94eb-a6c1b2acbc9f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:28.630326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 130/278] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:28.635852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394c9c1-f43c-44ef-94eb-a6c1b2acbc9f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:28.639710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d64954-8bb7-43e8-b389-33b9d00faf82 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:28.677171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791499e-bfe8-482b-8563-2938972945d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:28.677867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791499e-bfe8-482b-8563-2938972945d9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.678148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791499e-bfe8-482b-8563-2938972945d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.678417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791499e-bfe8-482b-8563-2938972945d9 None admin] PATH_INFO: `/v3/users/2ec0fef2e4f74553a0066cd6a0576706` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.693685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791499e-bfe8-482b-8563-2938972945d9 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec0fef2e4f74553a0066cd6a0576706)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:28.695184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791499e-bfe8-482b-8563-2938972945d9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:28.701564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 149/279] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:28 2022] GET /identity/v3/users/2ec0fef2e4f74553a0066cd6a0576706 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:28.701564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791499e-bfe8-482b-8563-2938972945d9 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:28.702613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:28.752716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:28.754723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394c9c1-f43c-44ef-94eb-a6c1b2acbc9f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:28.793668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1e71d-6a36-4e4d-8dcc-eeb5570b1ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:28.794288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1e71d-6a36-4e4d-8dcc-eeb5570b1ce2 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.794567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1e71d-6a36-4e4d-8dcc-eeb5570b1ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.794838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1e71d-6a36-4e4d-8dcc-eeb5570b1ce2 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.795493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1e71d-6a36-4e4d-8dcc-eeb5570b1ce2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:28.796961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1e71d-6a36-4e4d-8dcc-eeb5570b1ce2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:28.804081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791499e-bfe8-482b-8563-2938972945d9 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:28.816992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 131/280] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:38:28 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:38:28.818422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1e71d-6a36-4e4d-8dcc-eeb5570b1ce2 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:28.842659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa5524-2138-45d8-a9fa-fd6adfb4bb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:28.843222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa5524-2138-45d8-a9fa-fd6adfb4bb7f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.843501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa5524-2138-45d8-a9fa-fd6adfb4bb7f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa5524-2138-45d8-a9fa-fd6adfb4bb7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.855385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa5524-2138-45d8-a9fa-fd6adfb4bb7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-25fa5524-2138-45d8-a9fa-fd6adfb4bb7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:28.860405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:28.861366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:28.862202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:28.865663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 150/281] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:28.867674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d88275-cd9f-4ed3-92d6-9fe6a829da7f None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.868477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d88275-cd9f-4ed3-92d6-9fe6a829da7f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.870293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d88275-cd9f-4ed3-92d6-9fe6a829da7f None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.871724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 151/282] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:28.873656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d88275-cd9f-4ed3-92d6-9fe6a829da7f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:28.944119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1e71d-6a36-4e4d-8dcc-eeb5570b1ce2 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:28.954884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:28.984603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe6931-0e07-4063-8448-963af3867a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:28.985219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6931-0e07-4063-8448-963af3867a79 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.985507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6931-0e07-4063-8448-963af3867a79 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.986214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6931-0e07-4063-8448-963af3867a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:28.986791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe6931-0e07-4063-8448-963af3867a79 None admin] RBAC: Adding query filter params (role.id=47112770ffdc45638da090c45073888e, user.id=2ec0fef2e4f74553a0066cd6a0576706, scope.system=all) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:28.987233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe6931-0e07-4063-8448-963af3867a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:28.989036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe6931-0e07-4063-8448-963af3867a79 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:28.994569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 132/283] 10.222.0.46 () {56 vars in 1308 bytes} [Mon Oct 3 14:38:28 2022] GET /identity/v3/role_assignments?user.id=2ec0fef2e4f74553a0066cd6a0576706&scope.system=all&role.id=47112770ffdc45638da090c45073888e => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:28.995832 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78d87e-637b-41c8-ad2e-db60eb7c3258 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:28.996748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78d87e-637b-41c8-ad2e-db60eb7c3258 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:28.996748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78d87e-637b-41c8-ad2e-db60eb7c3258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:29.004490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d88275-cd9f-4ed3-92d6-9fe6a829da7f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:29.019026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:29.021217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6a78d87e-637b-41c8-ad2e-db60eb7c3258 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:29.022696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6a78d87e-637b-41c8-ad2e-db60eb7c3258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:29.052327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821ac7f-d890-4e28-968c-059d28f08280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:29.053195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821ac7f-d890-4e28-968c-059d28f08280 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:29.054840 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821ac7f-d890-4e28-968c-059d28f08280 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:29.054840 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821ac7f-d890-4e28-968c-059d28f08280 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:29.058773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821ac7f-d890-4e28-968c-059d28f08280 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:29.058773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821ac7f-d890-4e28-968c-059d28f08280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:29.058773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821ac7f-d890-4e28-968c-059d28f08280 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:29.074330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 152/284] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:29.080469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821ac7f-d890-4e28-968c-059d28f08280 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:29.115334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 133/285] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:29.201070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821ac7f-d890-4e28-968c-059d28f08280 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:29.236074 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6c4a3-a9f1-446d-b64e-a9b01966ba90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:29.236698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6c4a3-a9f1-446d-b64e-a9b01966ba90 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:29.236970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6c4a3-a9f1-446d-b64e-a9b01966ba90 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:29.237237 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6c4a3-a9f1-446d-b64e-a9b01966ba90 None admin] PATH_INFO: `/v3/users/f8cd5b61edb44dcdaf63f403b2c88a42` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:29.238382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6c4a3-a9f1-446d-b64e-a9b01966ba90 None admin] RBAC: Authorizing `identity:get_user(user_id=f8cd5b61edb44dcdaf63f403b2c88a42)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:29.239818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6c4a3-a9f1-446d-b64e-a9b01966ba90 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:29.241675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 153/286] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:29 2022] GET /identity/v3/users/f8cd5b61edb44dcdaf63f403b2c88a42 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:29.247833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78d87e-637b-41c8-ad2e-db60eb7c3258 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:29.355777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78d87e-637b-41c8-ad2e-db60eb7c3258 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:29.357432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:29.394057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bce4a2-78ea-46c3-aa67-bc65713dd76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:29.394772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bce4a2-78ea-46c3-aa67-bc65713dd76f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:29.395193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bce4a2-78ea-46c3-aa67-bc65713dd76f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:29.395561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bce4a2-78ea-46c3-aa67-bc65713dd76f None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:29.396775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bce4a2-78ea-46c3-aa67-bc65713dd76f None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:29.398480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bce4a2-78ea-46c3-aa67-bc65713dd76f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:29.400611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 134/287] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:29 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:29.406416 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6c4a3-a9f1-446d-b64e-a9b01966ba90 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:29.408720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bce4a2-78ea-46c3-aa67-bc65713dd76f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:29.469877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:29.582386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6c4a3-a9f1-446d-b64e-a9b01966ba90 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:29.602763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bce4a2-78ea-46c3-aa67-bc65713dd76f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:29.635156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49467fa-8180-43c4-a8d6-5ea115970a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:29.636480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49467fa-8180-43c4-a8d6-5ea115970a1f None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:29.636759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49467fa-8180-43c4-a8d6-5ea115970a1f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:29.637031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49467fa-8180-43c4-a8d6-5ea115970a1f None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23/users/f8cd5b61edb44dcdaf63f403b2c88a42/roles/8d46737ab3ff453a800fe6b47805eb5a` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:29.640392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49467fa-8180-43c4-a8d6-5ea115970a1f None admin] RBAC: Authorizing `identity:create_grant(project_id=805a9d6e523c465393a8f47773ab2b23, user_id=f8cd5b61edb44dcdaf63f403b2c88a42, role_id=8d46737ab3ff453a800fe6b47805eb5a)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:29.640643 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200bc28-f7d8-4d50-a47b-e9deb046816b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:29.641282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200bc28-f7d8-4d50-a47b-e9deb046816b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:29.641681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200bc28-f7d8-4d50-a47b-e9deb046816b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:29.641892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49467fa-8180-43c4-a8d6-5ea115970a1f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:29.641998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200bc28-f7d8-4d50-a47b-e9deb046816b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:29.652845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200bc28-f7d8-4d50-a47b-e9deb046816b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-5200bc28-f7d8-4d50-a47b-e9deb046816b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:29.658041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:29.659278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:29.660226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 135/288] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:29.667737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200bc28-f7d8-4d50-a47b-e9deb046816b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:29.673312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 154/289] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:38:29 2022] PUT /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23/users/f8cd5b61edb44dcdaf63f403b2c88a42/roles/8d46737ab3ff453a800fe6b47805eb5a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:29.702843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:29.782611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200bc28-f7d8-4d50-a47b-e9deb046816b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:29.821452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bdb44-5e83-47e3-9a4e-22d9bccf9c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:29.822107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bdb44-5e83-47e3-9a4e-22d9bccf9c8c None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:29.822389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bdb44-5e83-47e3-9a4e-22d9bccf9c8c None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:29.822663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bdb44-5e83-47e3-9a4e-22d9bccf9c8c None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:29.823389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bdb44-5e83-47e3-9a4e-22d9bccf9c8c None admin] RBAC: Adding query filter params (name=member) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:29.823696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bdb44-5e83-47e3-9a4e-22d9bccf9c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:29.825089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bdb44-5e83-47e3-9a4e-22d9bccf9c8c None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:29.834482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 136/290] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:29.840013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49467fa-8180-43c4-a8d6-5ea115970a1f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:29.994321 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:30.003632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49467fa-8180-43c4-a8d6-5ea115970a1f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:30.046470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a961e-cc86-4d18-a497-3ed54051076d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:30.047394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a961e-cc86-4d18-a497-3ed54051076d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:30.047847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a961e-cc86-4d18-a497-3ed54051076d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:30.048292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a961e-cc86-4d18-a497-3ed54051076d None admin] PATH_INFO: `/v3/users/4612079bfff84228bf010c4f0990cfcf` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:30.049693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a961e-cc86-4d18-a497-3ed54051076d None admin] RBAC: Authorizing `identity:get_user(user_id=4612079bfff84228bf010c4f0990cfcf)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:30.051301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a961e-cc86-4d18-a497-3ed54051076d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:30.053355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 155/291] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:29 2022] GET /identity/v3/users/4612079bfff84228bf010c4f0990cfcf => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:30.059277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bdb44-5e83-47e3-9a4e-22d9bccf9c8c None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:30.200905 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:30.218238 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bdb44-5e83-47e3-9a4e-22d9bccf9c8c None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:30.267170 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1e538-6b72-4169-98da-26d09e7b397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:30.268481 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e538-6b72-4169-98da-26d09e7b397c None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:30.268941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e538-6b72-4169-98da-26d09e7b397c None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:30.269386 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e538-6b72-4169-98da-26d09e7b397c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:30.373396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1e538-6b72-4169-98da-26d09e7b397c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:30.375168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1e538-6b72-4169-98da-26d09e7b397c None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:30.499618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:30.509856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 137/292] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:30.514889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a961e-cc86-4d18-a497-3ed54051076d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:30.638634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a961e-cc86-4d18-a497-3ed54051076d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:30.690177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41d175-f0e2-4f80-b199-b83a89f18935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:30.691106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41d175-f0e2-4f80-b199-b83a89f18935 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:30.691560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41d175-f0e2-4f80-b199-b83a89f18935 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:30.692005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41d175-f0e2-4f80-b199-b83a89f18935 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:30.693352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41d175-f0e2-4f80-b199-b83a89f18935 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:30.694980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41d175-f0e2-4f80-b199-b83a89f18935 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:30.696787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 156/293] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:30 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:30.703332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1e538-6b72-4169-98da-26d09e7b397c None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:30.820208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1e538-6b72-4169-98da-26d09e7b397c None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:30.865328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95246bdc-bb59-4786-87bb-387cd65da03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:30.866245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246bdc-bb59-4786-87bb-387cd65da03a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:30.866620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246bdc-bb59-4786-87bb-387cd65da03a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:30.866896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246bdc-bb59-4786-87bb-387cd65da03a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:30.867638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95246bdc-bb59-4786-87bb-387cd65da03a None admin] RBAC: Adding query filter params (role.id=47112770ffdc45638da090c45073888e, scope.project.id=805a9d6e523c465393a8f47773ab2b23, user.id=4612079bfff84228bf010c4f0990cfcf) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:30.867951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95246bdc-bb59-4786-87bb-387cd65da03a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:30.869403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95246bdc-bb59-4786-87bb-387cd65da03a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:30.875703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 138/294] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:30 2022] GET /identity/v3/role_assignments?user.id=4612079bfff84228bf010c4f0990cfcf&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=47112770ffdc45638da090c45073888e => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:31.444295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca81db-5176-4b68-a262-b88d3d6d6822 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:31.444686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca81db-5176-4b68-a262-b88d3d6d6822 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:31.444863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca81db-5176-4b68-a262-b88d3d6d6822 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:31.446463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 157/295] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:31.453771 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0a5db-4889-4284-94ad-7b5bc0a6c0ca None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:31.454220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0a5db-4889-4284-94ad-7b5bc0a6c0ca None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:31.454314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0a5db-4889-4284-94ad-7b5bc0a6c0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:31.476456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-bfb0a5db-4889-4284-94ad-7b5bc0a6c0ca None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:31.477682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-bfb0a5db-4889-4284-94ad-7b5bc0a6c0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:31.541922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:31.561440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 139/296] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:31.686174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58673e-3c62-40e1-8e7b-4ae84fdf514e None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:31.686678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58673e-3c62-40e1-8e7b-4ae84fdf514e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:31.686678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58673e-3c62-40e1-8e7b-4ae84fdf514e None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:31.688307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 158/297] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:31.693425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587fe0d-248a-4a21-b768-ecddca700a56 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:31.693746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587fe0d-248a-4a21-b768-ecddca700a56 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:31.694020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587fe0d-248a-4a21-b768-ecddca700a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:31.712501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-3587fe0d-248a-4a21-b768-ecddca700a56 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:31.714280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3587fe0d-248a-4a21-b768-ecddca700a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:31.788459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 140/298] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:31.833945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58673e-3c62-40e1-8e7b-4ae84fdf514e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:31.847655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a0495-0074-4cdf-a41f-675e7b48b26c None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:31.847953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a0495-0074-4cdf-a41f-675e7b48b26c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:31.848536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a0495-0074-4cdf-a41f-675e7b48b26c None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:31.849941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 141/299] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:31.855472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8336-3f34-4efe-9786-0e471b173267 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:31.855764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8336-3f34-4efe-9786-0e471b173267 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:31.856042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8336-3f34-4efe-9786-0e471b173267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:31.872708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:31.877042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-195b8336-3f34-4efe-9786-0e471b173267 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:31.878718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-195b8336-3f34-4efe-9786-0e471b173267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:31.945997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58673e-3c62-40e1-8e7b-4ae84fdf514e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:31.953625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 142/300] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:31.983674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956e4e6-a0b4-4799-961b-0e842be2546e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:31.984716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956e4e6-a0b4-4799-961b-0e842be2546e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:31.985189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956e4e6-a0b4-4799-961b-0e842be2546e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:31.985774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956e4e6-a0b4-4799-961b-0e842be2546e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:31.998487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956e4e6-a0b4-4799-961b-0e842be2546e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-8956e4e6-a0b4-4799-961b-0e842be2546e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:32.004017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:32.004877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:32.005668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:32.008262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 159/301] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:32.014275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b8336-3f34-4efe-9786-0e471b173267 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:32.059295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956e4e6-a0b4-4799-961b-0e842be2546e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:32.061898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:32.120033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b8336-3f34-4efe-9786-0e471b173267 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:32.158979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74320b5-cee5-4113-9ce9-f99d574c4e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:32.159610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74320b5-cee5-4113-9ce9-f99d574c4e07 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.159894 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74320b5-cee5-4113-9ce9-f99d574c4e07 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.160169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74320b5-cee5-4113-9ce9-f99d574c4e07 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.160989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74320b5-cee5-4113-9ce9-f99d574c4e07 None admin] RBAC: Adding query filter params (name=member) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:32.161445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74320b5-cee5-4113-9ce9-f99d574c4e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:32.162869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74320b5-cee5-4113-9ce9-f99d574c4e07 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:32.168177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:32.172356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 143/302] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:32.176417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956e4e6-a0b4-4799-961b-0e842be2546e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:32.177947 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74320b5-cee5-4113-9ce9-f99d574c4e07 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:32.216394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28163d12-014f-4ec3-994f-8707d65eda28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:32.217446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28163d12-014f-4ec3-994f-8707d65eda28 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.217995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28163d12-014f-4ec3-994f-8707d65eda28 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.218554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28163d12-014f-4ec3-994f-8707d65eda28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.230441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28163d12-014f-4ec3-994f-8707d65eda28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-28163d12-014f-4ec3-994f-8707d65eda28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:32.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:32.237127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:32.237972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:32.241859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 160/303] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:32.243262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28163d12-014f-4ec3-994f-8707d65eda28 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:32.303019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74320b5-cee5-4113-9ce9-f99d574c4e07 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:32.314005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:32.340597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461a899-88b8-4ef0-9e97-d2fa74827d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:32.341238 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461a899-88b8-4ef0-9e97-d2fa74827d62 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.341541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461a899-88b8-4ef0-9e97-d2fa74827d62 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.341831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461a899-88b8-4ef0-9e97-d2fa74827d62 None admin] PATH_INFO: `/v3/users/2ec0fef2e4f74553a0066cd6a0576706` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.342964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461a899-88b8-4ef0-9e97-d2fa74827d62 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec0fef2e4f74553a0066cd6a0576706)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:32.344407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461a899-88b8-4ef0-9e97-d2fa74827d62 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:32.346187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 144/304] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:32 2022] GET /identity/v3/users/2ec0fef2e4f74553a0066cd6a0576706 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:32.351430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461a899-88b8-4ef0-9e97-d2fa74827d62 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:32.354627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28163d12-014f-4ec3-994f-8707d65eda28 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:32.391086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83d325-5822-4cc5-bdf8-815e60c662aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:32.391638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83d325-5822-4cc5-bdf8-815e60c662aa None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.391916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83d325-5822-4cc5-bdf8-815e60c662aa None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.392192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83d325-5822-4cc5-bdf8-815e60c662aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.393718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83d325-5822-4cc5-bdf8-815e60c662aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:32.395537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83d325-5822-4cc5-bdf8-815e60c662aa None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:32.397709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 161/305] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:32.398460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83d325-5822-4cc5-bdf8-815e60c662aa None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:32.423293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:32.455226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461a899-88b8-4ef0-9e97-d2fa74827d62 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:32.488044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7595da-5567-441c-9eff-7b1475536483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:32.488649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7595da-5567-441c-9eff-7b1475536483 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.488930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7595da-5567-441c-9eff-7b1475536483 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.489201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7595da-5567-441c-9eff-7b1475536483 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.489899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7595da-5567-441c-9eff-7b1475536483 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:32.490207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7595da-5567-441c-9eff-7b1475536483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:32.491857 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7595da-5567-441c-9eff-7b1475536483 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:32.495822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83d325-5822-4cc5-bdf8-815e60c662aa None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:32.500148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 145/306] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:32.502775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacea18-3717-4ba4-9420-b0945cd23462 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.503064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacea18-3717-4ba4-9420-b0945cd23462 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.503339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacea18-3717-4ba4-9420-b0945cd23462 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.504819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 146/307] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:32.505573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacea18-3717-4ba4-9420-b0945cd23462 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:32.530301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8604b-1623-4abc-a5a1-a0ff351f4855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:32.530942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8604b-1623-4abc-a5a1-a0ff351f4855 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.531213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8604b-1623-4abc-a5a1-a0ff351f4855 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.531477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8604b-1623-4abc-a5a1-a0ff351f4855 None admin] PATH_INFO: `/v3/system/users/2ec0fef2e4f74553a0066cd6a0576706/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.532848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8604b-1623-4abc-a5a1-a0ff351f4855 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2ec0fef2e4f74553a0066cd6a0576706, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:32.534253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8604b-1623-4abc-a5a1-a0ff351f4855 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:32.543022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 162/308] 10.222.0.46 () {58 vars in 1334 bytes} [Mon Oct 3 14:38:32 2022] PUT /identity/v3/system/users/2ec0fef2e4f74553a0066cd6a0576706/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:32.543885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8604b-1623-4abc-a5a1-a0ff351f4855 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:32.548647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:32.621111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacea18-3717-4ba4-9420-b0945cd23462 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:32.643624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:32.660479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8604b-1623-4abc-a5a1-a0ff351f4855 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:32.662381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce222bb-41bc-4f91-9967-4d71194d8116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:32.662998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce222bb-41bc-4f91-9967-4d71194d8116 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.663285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce222bb-41bc-4f91-9967-4d71194d8116 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.663557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce222bb-41bc-4f91-9967-4d71194d8116 None admin] PATH_INFO: `/v3/groups` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.664204 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce222bb-41bc-4f91-9967-4d71194d8116 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:32.665701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce222bb-41bc-4f91-9967-4d71194d8116 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:32.683416 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 147/309] 10.222.0.46 () {60 vars in 1134 bytes} [Mon Oct 3 14:38:32 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:38:32.686909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ad7f1-f732-4a2c-aa58-9781c4090a05 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.687190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ad7f1-f732-4a2c-aa58-9781c4090a05 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.687466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ad7f1-f732-4a2c-aa58-9781c4090a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.712255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d43dc5-1b4b-4bb0-8e47-d5d95ec61b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:32.712937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d43dc5-1b4b-4bb0-8e47-d5d95ec61b0f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.713311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d43dc5-1b4b-4bb0-8e47-d5d95ec61b0f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.713686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d43dc5-1b4b-4bb0-8e47-d5d95ec61b0f None admin] PATH_INFO: `/v3/users/14696bb2b50b43b5b7150ea4b87f5a84` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.714014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-fc3ad7f1-f732-4a2c-aa58-9781c4090a05 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:32.715273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-fc3ad7f1-f732-4a2c-aa58-9781c4090a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:32.734248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d43dc5-1b4b-4bb0-8e47-d5d95ec61b0f None admin] RBAC: Authorizing `identity:get_user(user_id=14696bb2b50b43b5b7150ea4b87f5a84)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:32.736134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d43dc5-1b4b-4bb0-8e47-d5d95ec61b0f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:32.738322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 163/310] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:32 2022] GET /identity/v3/users/14696bb2b50b43b5b7150ea4b87f5a84 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:32.744790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d43dc5-1b4b-4bb0-8e47-d5d95ec61b0f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:32.822947 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 148/311] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:32.877486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d43dc5-1b4b-4bb0-8e47-d5d95ec61b0f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:32.921889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4dff2-a15d-43f6-af87-e854f16ce701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:32.922778 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4dff2-a15d-43f6-af87-e854f16ce701 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:32.923194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4dff2-a15d-43f6-af87-e854f16ce701 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:32.923598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4dff2-a15d-43f6-af87-e854f16ce701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:32.924738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4dff2-a15d-43f6-af87-e854f16ce701 None admin] RBAC: Adding query filter params (role.id=8d46737ab3ff453a800fe6b47805eb5a, user.id=14696bb2b50b43b5b7150ea4b87f5a84, scope.system=all) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:32.925202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4dff2-a15d-43f6-af87-e854f16ce701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:32.927501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4dff2-a15d-43f6-af87-e854f16ce701 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:32.936679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 164/312] 10.222.0.46 () {56 vars in 1308 bytes} [Mon Oct 3 14:38:32 2022] GET /identity/v3/role_assignments?user.id=14696bb2b50b43b5b7150ea4b87f5a84&scope.system=all&role.id=8d46737ab3ff453a800fe6b47805eb5a => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:33.107136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ad7f1-f732-4a2c-aa58-9781c4090a05 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:33.137528 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:33.212248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ad7f1-f732-4a2c-aa58-9781c4090a05 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:33.248645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795912f2-3613-4499-973b-648b2b3feb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:33.249744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795912f2-3613-4499-973b-648b2b3feb60 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:33.250223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795912f2-3613-4499-973b-648b2b3feb60 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:33.250713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795912f2-3613-4499-973b-648b2b3feb60 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:33.262970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795912f2-3613-4499-973b-648b2b3feb60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-795912f2-3613-4499-973b-648b2b3feb60 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:33.268259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:33.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:33.269955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:33.272674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 149/313] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:33.284398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4dff2-a15d-43f6-af87-e854f16ce701 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:33.310980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:33.396534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4dff2-a15d-43f6-af87-e854f16ce701 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:33.433297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726ba0a-401c-4947-b490-da160e303533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:33.433932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726ba0a-401c-4947-b490-da160e303533 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:33.434208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726ba0a-401c-4947-b490-da160e303533 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:33.434474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726ba0a-401c-4947-b490-da160e303533 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:33.435137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726ba0a-401c-4947-b490-da160e303533 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:33.435436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726ba0a-401c-4947-b490-da160e303533 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:33.436813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726ba0a-401c-4947-b490-da160e303533 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:33.445709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 165/314] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:33.451153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795912f2-3613-4499-973b-648b2b3feb60 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:33.546157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:33.558090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795912f2-3613-4499-973b-648b2b3feb60 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:33.592942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06e3d-cd4b-4ed6-9f5e-82fdc0c24d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:33.593602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06e3d-cd4b-4ed6-9f5e-82fdc0c24d2e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:33.593886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06e3d-cd4b-4ed6-9f5e-82fdc0c24d2e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:33.594157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06e3d-cd4b-4ed6-9f5e-82fdc0c24d2e None admin] PATH_INFO: `/v3/users/f8cd5b61edb44dcdaf63f403b2c88a42` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:33.595319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd06e3d-cd4b-4ed6-9f5e-82fdc0c24d2e None admin] RBAC: Authorizing `identity:get_user(user_id=f8cd5b61edb44dcdaf63f403b2c88a42)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:33.596778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd06e3d-cd4b-4ed6-9f5e-82fdc0c24d2e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:33.598839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 150/315] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:33 2022] GET /identity/v3/users/f8cd5b61edb44dcdaf63f403b2c88a42 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:33.604892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726ba0a-401c-4947-b490-da160e303533 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:33.678975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:33.713968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726ba0a-401c-4947-b490-da160e303533 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:33.749600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb0b40-fff0-4132-9b1b-f9d641ea2588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:33.750535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb0b40-fff0-4132-9b1b-f9d641ea2588 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:33.751032 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb0b40-fff0-4132-9b1b-f9d641ea2588 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:33.751491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb0b40-fff0-4132-9b1b-f9d641ea2588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:33.821901 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb0b40-fff0-4132-9b1b-f9d641ea2588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:33.824270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb0b40-fff0-4132-9b1b-f9d641ea2588 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:33.899403 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:33.931513 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 166/316] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:33.937721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06e3d-cd4b-4ed6-9f5e-82fdc0c24d2e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:34.039036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06e3d-cd4b-4ed6-9f5e-82fdc0c24d2e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:34.073595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3f834-ba1d-490a-8b64-ac0c45236dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:34.074153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3f834-ba1d-490a-8b64-ac0c45236dc2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:34.074432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3f834-ba1d-490a-8b64-ac0c45236dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:34.074703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3f834-ba1d-490a-8b64-ac0c45236dc2 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:34.075824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3f834-ba1d-490a-8b64-ac0c45236dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:34.077273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3f834-ba1d-490a-8b64-ac0c45236dc2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:34.079212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 151/317] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:33 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:34.085014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb0b40-fff0-4132-9b1b-f9d641ea2588 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:34.186186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb0b40-fff0-4132-9b1b-f9d641ea2588 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:34.221243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e3313-43e8-473e-a674-edad29092d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:34.222214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e3313-43e8-473e-a674-edad29092d86 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:34.222707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e3313-43e8-473e-a674-edad29092d86 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:34.223166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e3313-43e8-473e-a674-edad29092d86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:34.224078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e3313-43e8-473e-a674-edad29092d86 None admin] RBAC: Adding query filter params (role.id=8d46737ab3ff453a800fe6b47805eb5a, scope.project.id=805a9d6e523c465393a8f47773ab2b23, user.id=f8cd5b61edb44dcdaf63f403b2c88a42) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:34.224591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e3313-43e8-473e-a674-edad29092d86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:34.226228 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e3313-43e8-473e-a674-edad29092d86 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:34.233405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 167/318] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:38:34 2022] GET /identity/v3/role_assignments?user.id=f8cd5b61edb44dcdaf63f403b2c88a42&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=8d46737ab3ff453a800fe6b47805eb5a => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:34.756761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac835330-d743-4ba8-8e48-90aea722d1a5 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:34.757275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac835330-d743-4ba8-8e48-90aea722d1a5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:34.757377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac835330-d743-4ba8-8e48-90aea722d1a5 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:34.758820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 152/319] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:34.763842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a127-c6cc-44e9-8b43-56b4ba3394a5 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:34.764209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a127-c6cc-44e9-8b43-56b4ba3394a5 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:34.764390 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a127-c6cc-44e9-8b43-56b4ba3394a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:34.782819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d570a127-c6cc-44e9-8b43-56b4ba3394a5 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:34.784002 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d570a127-c6cc-44e9-8b43-56b4ba3394a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:34.812784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:34.851025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 168/320] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:35.001780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e335b9-4cef-40a8-b45f-a0b56cfa6404 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.002126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e335b9-4cef-40a8-b45f-a0b56cfa6404 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.002321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e335b9-4cef-40a8-b45f-a0b56cfa6404 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.003788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 153/321] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:35.008908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3393f-0fc6-4324-a804-f65b1af67ef8 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.009134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3393f-0fc6-4324-a804-f65b1af67ef8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.009400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3393f-0fc6-4324-a804-f65b1af67ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.028971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c6a3393f-0fc6-4324-a804-f65b1af67ef8 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:35.030211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c6a3393f-0fc6-4324-a804-f65b1af67ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:35.100577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 169/322] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:35.122686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e335b9-4cef-40a8-b45f-a0b56cfa6404 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:35.146237 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198215-aea7-4867-b920-1b5a270aa5fb None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.146527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198215-aea7-4867-b920-1b5a270aa5fb None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.147071 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198215-aea7-4867-b920-1b5a270aa5fb None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.148714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 170/323] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:38:35.149165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:35.153919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2e78b-e038-49ad-8599-25d00ca27457 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.154207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2e78b-e038-49ad-8599-25d00ca27457 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.154481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2e78b-e038-49ad-8599-25d00ca27457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.172553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b3d2e78b-e038-49ad-8599-25d00ca27457 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:35.173761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b3d2e78b-e038-49ad-8599-25d00ca27457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:35.232453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e335b9-4cef-40a8-b45f-a0b56cfa6404 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:35.242248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 171/324] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:35.269125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737fbe6-e5ae-4adf-937e-9868c676385b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:35.270037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737fbe6-e5ae-4adf-937e-9868c676385b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.270525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737fbe6-e5ae-4adf-937e-9868c676385b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.270972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737fbe6-e5ae-4adf-937e-9868c676385b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.283342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737fbe6-e5ae-4adf-937e-9868c676385b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-f737fbe6-e5ae-4adf-937e-9868c676385b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:35.288194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:35.289057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:35.289892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:35.294756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 154/325] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:35.297805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2e78b-e038-49ad-8599-25d00ca27457 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:35.325560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:35.367994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737fbe6-e5ae-4adf-937e-9868c676385b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:35.403985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2e78b-e038-49ad-8599-25d00ca27457 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:35.441624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892b53a-55e4-43b8-914b-8a036ef9bcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:35.442219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892b53a-55e4-43b8-914b-8a036ef9bcc3 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.442494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892b53a-55e4-43b8-914b-8a036ef9bcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.442761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892b53a-55e4-43b8-914b-8a036ef9bcc3 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.443414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892b53a-55e4-43b8-914b-8a036ef9bcc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:35.443710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892b53a-55e4-43b8-914b-8a036ef9bcc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:35.445062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892b53a-55e4-43b8-914b-8a036ef9bcc3 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:35.453212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 172/326] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:35.458860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892b53a-55e4-43b8-914b-8a036ef9bcc3 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:35.463907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:35.485864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737fbe6-e5ae-4adf-937e-9868c676385b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:35.522718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba85bd-172f-4842-8a1e-62000ea3d489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:35.523277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba85bd-172f-4842-8a1e-62000ea3d489 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.523614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba85bd-172f-4842-8a1e-62000ea3d489 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.523821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba85bd-172f-4842-8a1e-62000ea3d489 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.524987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba85bd-172f-4842-8a1e-62000ea3d489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:35.526986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba85bd-172f-4842-8a1e-62000ea3d489 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:35.528593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 155/327] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:35.529350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba85bd-172f-4842-8a1e-62000ea3d489 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:35.547330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:35.559871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892b53a-55e4-43b8-914b-8a036ef9bcc3 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:35.594634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3aeaf9-32f0-4bcb-9dfd-91bc8291b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:35.595246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aeaf9-32f0-4bcb-9dfd-91bc8291b266 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.595521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aeaf9-32f0-4bcb-9dfd-91bc8291b266 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.595783 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aeaf9-32f0-4bcb-9dfd-91bc8291b266 None admin] PATH_INFO: `/v3/users/2ec0fef2e4f74553a0066cd6a0576706` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.596884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3aeaf9-32f0-4bcb-9dfd-91bc8291b266 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec0fef2e4f74553a0066cd6a0576706)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:35.598336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3aeaf9-32f0-4bcb-9dfd-91bc8291b266 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:35.599877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 173/328] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:35 2022] GET /identity/v3/users/2ec0fef2e4f74553a0066cd6a0576706 => generated 293 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:35.602311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3aeaf9-32f0-4bcb-9dfd-91bc8291b266 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:35.629474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba85bd-172f-4842-8a1e-62000ea3d489 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:35.662162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08507447-6846-4d11-9740-235c9f07fc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:35.662782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08507447-6846-4d11-9740-235c9f07fc9b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.663056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08507447-6846-4d11-9740-235c9f07fc9b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.663323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08507447-6846-4d11-9740-235c9f07fc9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.674480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08507447-6846-4d11-9740-235c9f07fc9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-08507447-6846-4d11-9740-235c9f07fc9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:35.679235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:35.680109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:35.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 156/329] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:35.681601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08507447-6846-4d11-9740-235c9f07fc9b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:35.695307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:35.706593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3aeaf9-32f0-4bcb-9dfd-91bc8291b266 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:35.739560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fa908-b15d-4402-a2e3-fc2fc7b1f3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:35.740182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fa908-b15d-4402-a2e3-fc2fc7b1f3d2 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.740479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fa908-b15d-4402-a2e3-fc2fc7b1f3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.741040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fa908-b15d-4402-a2e3-fc2fc7b1f3d2 None admin] PATH_INFO: `/v3/groups` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.741758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fa908-b15d-4402-a2e3-fc2fc7b1f3d2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:35.743259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fa908-b15d-4402-a2e3-fc2fc7b1f3d2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:35.760034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 174/330] 10.222.0.46 () {60 vars in 1134 bytes} [Mon Oct 3 14:38:35 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:38:35.761010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fa908-b15d-4402-a2e3-fc2fc7b1f3d2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:35.780943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:35.789411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08507447-6846-4d11-9740-235c9f07fc9b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:35.829120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0da703-c74b-4a99-9518-46c31d37d3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:35.829750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0da703-c74b-4a99-9518-46c31d37d3e4 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.830033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0da703-c74b-4a99-9518-46c31d37d3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.830321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0da703-c74b-4a99-9518-46c31d37d3e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.831262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0da703-c74b-4a99-9518-46c31d37d3e4 None admin] RBAC: Adding query filter params (role.id=47112770ffdc45638da090c45073888e, user.id=2ec0fef2e4f74553a0066cd6a0576706, scope.system=all) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:35.831401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0da703-c74b-4a99-9518-46c31d37d3e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:35.832900 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0da703-c74b-4a99-9518-46c31d37d3e4 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:35.846582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 157/331] 10.222.0.46 () {56 vars in 1308 bytes} [Mon Oct 3 14:38:35 2022] GET /identity/v3/role_assignments?user.id=2ec0fef2e4f74553a0066cd6a0576706&scope.system=all&role.id=47112770ffdc45638da090c45073888e => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:35.879739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fa908-b15d-4402-a2e3-fc2fc7b1f3d2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:35.919509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d496a-31ad-4c90-8828-59a3619eb805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:35.920137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d496a-31ad-4c90-8828-59a3619eb805 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:35.920413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d496a-31ad-4c90-8828-59a3619eb805 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:35.920681 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d496a-31ad-4c90-8828-59a3619eb805 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:35.921373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d496a-31ad-4c90-8828-59a3619eb805 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:35.921715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d496a-31ad-4c90-8828-59a3619eb805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:35.923152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d496a-31ad-4c90-8828-59a3619eb805 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:35.937422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 175/332] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:35.939233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0da703-c74b-4a99-9518-46c31d37d3e4 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:36.020449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:36.056208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0da703-c74b-4a99-9518-46c31d37d3e4 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:36.098880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada492e-d80d-4d2c-98f0-aa6cccce5fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:36.099576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada492e-d80d-4d2c-98f0-aa6cccce5fe2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:36.099859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada492e-d80d-4d2c-98f0-aa6cccce5fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:36.100131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada492e-d80d-4d2c-98f0-aa6cccce5fe2 None admin] PATH_INFO: `/v3/users/14696bb2b50b43b5b7150ea4b87f5a84` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:36.101272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada492e-d80d-4d2c-98f0-aa6cccce5fe2 None admin] RBAC: Authorizing `identity:get_user(user_id=14696bb2b50b43b5b7150ea4b87f5a84)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:36.102874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada492e-d80d-4d2c-98f0-aa6cccce5fe2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:36.105269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 158/333] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:35 2022] GET /identity/v3/users/14696bb2b50b43b5b7150ea4b87f5a84 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:36.110935 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d496a-31ad-4c90-8828-59a3619eb805 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:36.201215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:36.226937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d496a-31ad-4c90-8828-59a3619eb805 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:36.267926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb81cd-85be-4c3e-a10e-938d65cd2765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:36.268582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb81cd-85be-4c3e-a10e-938d65cd2765 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:36.268860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb81cd-85be-4c3e-a10e-938d65cd2765 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:36.269138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb81cd-85be-4c3e-a10e-938d65cd2765 None admin] PATH_INFO: `/v3/system/users/14696bb2b50b43b5b7150ea4b87f5a84/roles/8d46737ab3ff453a800fe6b47805eb5a` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:36.270597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb81cd-85be-4c3e-a10e-938d65cd2765 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=14696bb2b50b43b5b7150ea4b87f5a84, role_id=8d46737ab3ff453a800fe6b47805eb5a)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:36.271989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb81cd-85be-4c3e-a10e-938d65cd2765 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:36.284679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 176/334] 10.222.0.46 () {58 vars in 1334 bytes} [Mon Oct 3 14:38:36 2022] PUT /identity/v3/system/users/14696bb2b50b43b5b7150ea4b87f5a84/roles/8d46737ab3ff453a800fe6b47805eb5a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:38.307240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667df09-0a66-4e66-90cd-e13a1d4a1d7b None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:38.308115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667df09-0a66-4e66-90cd-e13a1d4a1d7b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:38.308621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667df09-0a66-4e66-90cd-e13a1d4a1d7b None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:38.310607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 159/335] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:38.315982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b091b60-c6a8-491d-b923-3ef85565ced2 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:38.316475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b091b60-c6a8-491d-b923-3ef85565ced2 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:38.316578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b091b60-c6a8-491d-b923-3ef85565ced2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:38.346906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-1b091b60-c6a8-491d-b923-3ef85565ced2 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:38.348137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1b091b60-c6a8-491d-b923-3ef85565ced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:38.443827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 177/336] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:38.539999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151581f-fb2a-49ca-9fec-fb5314cc405d None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:38.540649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151581f-fb2a-49ca-9fec-fb5314cc405d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:38.541197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151581f-fb2a-49ca-9fec-fb5314cc405d None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:38.543032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 160/337] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:38.548759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbfc54-3594-4052-a9d9-17c40be5da9c None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:38.548994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbfc54-3594-4052-a9d9-17c40be5da9c None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:38.549248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbfc54-3594-4052-a9d9-17c40be5da9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:38.578749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-4dbbfc54-3594-4052-a9d9-17c40be5da9c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:38.579970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4dbbfc54-3594-4052-a9d9-17c40be5da9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:38.677987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 178/338] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:38.702033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd207b9d-b2d0-4ecb-8e1e-fc8ef5146029 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:38.702827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd207b9d-b2d0-4ecb-8e1e-fc8ef5146029 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:38.703321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd207b9d-b2d0-4ecb-8e1e-fc8ef5146029 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:38.705098 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 161/339] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:38.710657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d75b42-7253-40ae-b9dd-526ea6372411 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:38.710949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d75b42-7253-40ae-b9dd-526ea6372411 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:38.711217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d75b42-7253-40ae-b9dd-526ea6372411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:38.719199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:38.735178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-94d75b42-7253-40ae-b9dd-526ea6372411 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:38.736350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-94d75b42-7253-40ae-b9dd-526ea6372411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:38.828053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 179/340] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:38.961759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd207b9d-b2d0-4ecb-8e1e-fc8ef5146029 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:39.080965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd207b9d-b2d0-4ecb-8e1e-fc8ef5146029 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:39.122865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95037a-d283-4a18-a05b-ede232d56213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:39.123852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95037a-d283-4a18-a05b-ede232d56213 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:39.124328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95037a-d283-4a18-a05b-ede232d56213 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:39.124820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95037a-d283-4a18-a05b-ede232d56213 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:39.136649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:39.141472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95037a-d283-4a18-a05b-ede232d56213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-4b95037a-d283-4a18-a05b-ede232d56213 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:39.146759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:39.147630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:39.148410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:39.154211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 162/341] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:39.157926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d75b42-7253-40ae-b9dd-526ea6372411 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:39.167359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95037a-d283-4a18-a05b-ede232d56213 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:39.322310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95037a-d283-4a18-a05b-ede232d56213 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:39.357743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7a8bb-df62-4f13-b707-74d3c49be7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:39.358307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7a8bb-df62-4f13-b707-74d3c49be7ec None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:39.358584 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7a8bb-df62-4f13-b707-74d3c49be7ec None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:39.358853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7a8bb-df62-4f13-b707-74d3c49be7ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:39.370002 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d75b42-7253-40ae-b9dd-526ea6372411 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:39.370459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7a8bb-df62-4f13-b707-74d3c49be7ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-60b7a8bb-df62-4f13-b707-74d3c49be7ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:39.375340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:39.376196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:39.376974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 163/342] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:39.383890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7a8bb-df62-4f13-b707-74d3c49be7ec None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:39.432063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:39.453271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a10f4-a602-4e38-8764-9553e2a9821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:39.454306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a10f4-a602-4e38-8764-9553e2a9821c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:39.454884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a10f4-a602-4e38-8764-9553e2a9821c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:39.455348 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a10f4-a602-4e38-8764-9553e2a9821c None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:39.456274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a10f4-a602-4e38-8764-9553e2a9821c None admin] RBAC: Adding query filter params (name=member) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:39.456771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a10f4-a602-4e38-8764-9553e2a9821c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:39.458435 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a10f4-a602-4e38-8764-9553e2a9821c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:39.472233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:39.473405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 180/343] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:39.474254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a10f4-a602-4e38-8764-9553e2a9821c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:39.514981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7a8bb-df62-4f13-b707-74d3c49be7ec None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:39.554604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12b409-3fc5-4298-b32f-05f144753cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:39.555890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12b409-3fc5-4298-b32f-05f144753cbc None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:39.557219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12b409-3fc5-4298-b32f-05f144753cbc None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:39.557742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12b409-3fc5-4298-b32f-05f144753cbc None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:39.558968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12b409-3fc5-4298-b32f-05f144753cbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:39.560431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12b409-3fc5-4298-b32f-05f144753cbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:39.562314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12b409-3fc5-4298-b32f-05f144753cbc None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:39.582892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 164/344] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:39.584903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12b409-3fc5-4298-b32f-05f144753cbc None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:39.677391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a10f4-a602-4e38-8764-9553e2a9821c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:39.722153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12b409-3fc5-4298-b32f-05f144753cbc None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:39.730502 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdbe34-b6ae-4397-acf8-4f3a935a46cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:39.731153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdbe34-b6ae-4397-acf8-4f3a935a46cb None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:39.731431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdbe34-b6ae-4397-acf8-4f3a935a46cb None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:39.731701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdbe34-b6ae-4397-acf8-4f3a935a46cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:39.745389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:39.748332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdbe34-b6ae-4397-acf8-4f3a935a46cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-72cdbe34-b6ae-4397-acf8-4f3a935a46cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:39.753846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:39.754740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:39.755610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 181/345] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:39.756269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdbe34-b6ae-4397-acf8-4f3a935a46cb None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:39.771091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922cb38-9bbd-4f0b-8cfb-23dd17c0de24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:39.772053 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922cb38-9bbd-4f0b-8cfb-23dd17c0de24 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:39.772570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922cb38-9bbd-4f0b-8cfb-23dd17c0de24 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:39.773040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922cb38-9bbd-4f0b-8cfb-23dd17c0de24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:39.833928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:39.860253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922cb38-9bbd-4f0b-8cfb-23dd17c0de24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:39.861755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922cb38-9bbd-4f0b-8cfb-23dd17c0de24 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:39.876144 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdbe34-b6ae-4397-acf8-4f3a935a46cb None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:39.916367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e92ae-5471-496f-ac58-0052ddf28d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:39.917079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e92ae-5471-496f-ac58-0052ddf28d5a None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:39.917360 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e92ae-5471-496f-ac58-0052ddf28d5a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:39.917655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e92ae-5471-496f-ac58-0052ddf28d5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:39.985531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 165/346] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:39.986898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc9a5c-1d97-4608-b83c-b788ae8daf49 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:39.987187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc9a5c-1d97-4608-b83c-b788ae8daf49 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:39.987461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc9a5c-1d97-4608-b83c-b788ae8daf49 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:39.988878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 166/347] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:39.989422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc9a5c-1d97-4608-b83c-b788ae8daf49 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:40.012334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e92ae-5471-496f-ac58-0052ddf28d5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:40.013880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e92ae-5471-496f-ac58-0052ddf28d5a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:40.015821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:40.095415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:40.142136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc9a5c-1d97-4608-b83c-b788ae8daf49 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:40.155876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 182/348] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:40.157175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e92ae-5471-496f-ac58-0052ddf28d5a None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:40.189877 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f65150-5b94-470e-ba08-dc53f8e8cbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:40.190493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f65150-5b94-470e-ba08-dc53f8e8cbc2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.190767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f65150-5b94-470e-ba08-dc53f8e8cbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.191032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f65150-5b94-470e-ba08-dc53f8e8cbc2 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.191715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f65150-5b94-470e-ba08-dc53f8e8cbc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:40.191994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f65150-5b94-470e-ba08-dc53f8e8cbc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:40.193359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f65150-5b94-470e-ba08-dc53f8e8cbc2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:40.207685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 167/349] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:40.209004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158eb32-8756-4214-a3d7-b2f2c293c819 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.209294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158eb32-8756-4214-a3d7-b2f2c293c819 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.209590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158eb32-8756-4214-a3d7-b2f2c293c819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.230306 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b158eb32-8756-4214-a3d7-b2f2c293c819 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:40.231683 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b158eb32-8756-4214-a3d7-b2f2c293c819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:40.280113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e92ae-5471-496f-ac58-0052ddf28d5a None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:40.319539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 168/350] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:40.320611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158eb32-8756-4214-a3d7-b2f2c293c819 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:40.335795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:40.342640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6310a5-5e0b-45e7-a022-6b077945e18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:40.343393 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6310a5-5e0b-45e7-a022-6b077945e18b None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.343768 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6310a5-5e0b-45e7-a022-6b077945e18b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.344133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6310a5-5e0b-45e7-a022-6b077945e18b None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.345884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6310a5-5e0b-45e7-a022-6b077945e18b None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:40.347729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6310a5-5e0b-45e7-a022-6b077945e18b None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:40.350221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 183/351] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:40 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:40.352163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906970af-f87d-4dcf-aca3-d4655900e1fb None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.352582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906970af-f87d-4dcf-aca3-d4655900e1fb None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.352976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906970af-f87d-4dcf-aca3-d4655900e1fb None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.361041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 184/352] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:40.362001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906970af-f87d-4dcf-aca3-d4655900e1fb None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:40.389917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:40.451684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158eb32-8756-4214-a3d7-b2f2c293c819 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:40.493754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aa4fa-6b3f-4236-ab91-8114339a99f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:40.494441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aa4fa-6b3f-4236-ab91-8114339a99f8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.494840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aa4fa-6b3f-4236-ab91-8114339a99f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.495250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aa4fa-6b3f-4236-ab91-8114339a99f8 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.496509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0aa4fa-6b3f-4236-ab91-8114339a99f8 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:40.498279 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0aa4fa-6b3f-4236-ab91-8114339a99f8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:40.500539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 169/353] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:40 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:40.501546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906970af-f87d-4dcf-aca3-d4655900e1fb None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:40.505845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aa4fa-6b3f-4236-ab91-8114339a99f8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:40.547743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622829e3-c8f6-4ca8-835e-e051019098b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:40.548277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622829e3-c8f6-4ca8-835e-e051019098b1 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.548551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622829e3-c8f6-4ca8-835e-e051019098b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.548818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622829e3-c8f6-4ca8-835e-e051019098b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.615494 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:40.639729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aa4fa-6b3f-4236-ab91-8114339a99f8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:40.644344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622829e3-c8f6-4ca8-835e-e051019098b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:40.645832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622829e3-c8f6-4ca8-835e-e051019098b1 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:40.677844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:40.678352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490af897-e58f-4cc9-9681-518edfdbc9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:40.679485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490af897-e58f-4cc9-9681-518edfdbc9eb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.679966 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490af897-e58f-4cc9-9681-518edfdbc9eb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.680336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490af897-e58f-4cc9-9681-518edfdbc9eb None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.688914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490af897-e58f-4cc9-9681-518edfdbc9eb None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:40.690430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490af897-e58f-4cc9-9681-518edfdbc9eb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:40.692096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 170/354] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:40 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:40.694322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba42994-21c8-4ba4-b175-e5e1a51bd1c2 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.694609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba42994-21c8-4ba4-b175-e5e1a51bd1c2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.694884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba42994-21c8-4ba4-b175-e5e1a51bd1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.714855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-0ba42994-21c8-4ba4-b175-e5e1a51bd1c2 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:40.716055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0ba42994-21c8-4ba4-b175-e5e1a51bd1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:40.763247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 185/355] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:40.765292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622829e3-c8f6-4ca8-835e-e051019098b1 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:40.796483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 171/356] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:40.798138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba42994-21c8-4ba4-b175-e5e1a51bd1c2 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:40.892478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:40.897949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622829e3-c8f6-4ca8-835e-e051019098b1 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:40.924897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:40.927224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba42994-21c8-4ba4-b175-e5e1a51bd1c2 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:40.937489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8ea17-6f22-484c-81e6-c4c4a4b2caa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:40.938334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8ea17-6f22-484c-81e6-c4c4a4b2caa7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.938777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8ea17-6f22-484c-81e6-c4c4a4b2caa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.939306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8ea17-6f22-484c-81e6-c4c4a4b2caa7 None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.940488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8ea17-6f22-484c-81e6-c4c4a4b2caa7 None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:40.942104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8ea17-6f22-484c-81e6-c4c4a4b2caa7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:40.944198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 186/357] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:40 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:40.945198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442b76c-9e47-43b0-98b7-898e26833f01 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.945474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442b76c-9e47-43b0-98b7-898e26833f01 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.945764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442b76c-9e47-43b0-98b7-898e26833f01 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.947594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 187/358] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:40.948142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442b76c-9e47-43b0-98b7-898e26833f01 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:40.977790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c7c63-176b-4756-86a2-e6cc5dfa0ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:40.978966 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c7c63-176b-4756-86a2-e6cc5dfa0ede None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:40.979247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c7c63-176b-4756-86a2-e6cc5dfa0ede None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:40.979679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c7c63-176b-4756-86a2-e6cc5dfa0ede None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:40.995674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c7c63-176b-4756-86a2-e6cc5dfa0ede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-be1c7c63-176b-4756-86a2-e6cc5dfa0ede None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:41.000460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:41.001303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 172/359] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:41.003749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c7c63-176b-4756-86a2-e6cc5dfa0ede None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:41.075706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442b76c-9e47-43b0-98b7-898e26833f01 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:41.117101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a6f12-926c-4dcb-89c9-fd2bbffe3647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:41.117863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a6f12-926c-4dcb-89c9-fd2bbffe3647 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:41.118146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a6f12-926c-4dcb-89c9-fd2bbffe3647 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:41.118398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a6f12-926c-4dcb-89c9-fd2bbffe3647 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:41.119224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a6f12-926c-4dcb-89c9-fd2bbffe3647 None admin] RBAC: Adding query filter params (group.id=36e9ba2eae5542fd84e9624ca2971649, role.id=47112770ffdc45638da090c45073888e, scope.project.id=130f64a6280e4b83bf885edf8986e895) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:41.119525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a6f12-926c-4dcb-89c9-fd2bbffe3647 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:41.120941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a6f12-926c-4dcb-89c9-fd2bbffe3647 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:41.126340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c7c63-176b-4756-86a2-e6cc5dfa0ede None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:41.128426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 188/360] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 3 14:38:40 2022] GET /identity/v3/role_assignments?group.id=36e9ba2eae5542fd84e9624ca2971649&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=47112770ffdc45638da090c45073888e => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:41.129117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a6f12-926c-4dcb-89c9-fd2bbffe3647 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:41.182256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd8666-2f61-42f3-9781-1106128f1a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:41.182962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd8666-2f61-42f3-9781-1106128f1a95 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:41.183309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd8666-2f61-42f3-9781-1106128f1a95 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:41.183578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd8666-2f61-42f3-9781-1106128f1a95 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:41.184850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd8666-2f61-42f3-9781-1106128f1a95 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:41.189033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd8666-2f61-42f3-9781-1106128f1a95 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:41.189033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 173/361] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:41 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:41.190033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab81af-08cf-4c3e-9c87-32c83d888596 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:41.190162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab81af-08cf-4c3e-9c87-32c83d888596 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:41.190419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab81af-08cf-4c3e-9c87-32c83d888596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:41.197718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:41.212747 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-55ab81af-08cf-4c3e-9c87-32c83d888596 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:41.214038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-55ab81af-08cf-4c3e-9c87-32c83d888596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:41.215003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:41.274848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a6f12-926c-4dcb-89c9-fd2bbffe3647 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:41.310774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 174/362] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:41.312509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab81af-08cf-4c3e-9c87-32c83d888596 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:41.320745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7bfb-d968-427c-a87b-7612f6ff1452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:41.321525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7bfb-d968-427c-a87b-7612f6ff1452 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:41.321826 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7bfb-d968-427c-a87b-7612f6ff1452 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:41.322180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7bfb-d968-427c-a87b-7612f6ff1452 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:41.323074 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f7bfb-d968-427c-a87b-7612f6ff1452 None admin] RBAC: Adding query filter params (group.id=36e9ba2eae5542fd84e9624ca2971649, role.id=8cd21d388193490bac3aa386bcc18763, scope.project.id=130f64a6280e4b83bf885edf8986e895) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:41.323427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f7bfb-d968-427c-a87b-7612f6ff1452 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:41.325026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f7bfb-d968-427c-a87b-7612f6ff1452 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:41.332623 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 189/363] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 3 14:38:41 2022] GET /identity/v3/role_assignments?group.id=36e9ba2eae5542fd84e9624ca2971649&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=8cd21d388193490bac3aa386bcc18763 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:41.333272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7bfb-d968-427c-a87b-7612f6ff1452 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:41.448618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7bfb-d968-427c-a87b-7612f6ff1452 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:41.476052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab81af-08cf-4c3e-9c87-32c83d888596 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:41.481880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:41.522997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515658dd-af46-4d8b-8a8b-71f5af0f530b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:41.523644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515658dd-af46-4d8b-8a8b-71f5af0f530b None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:41.523920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515658dd-af46-4d8b-8a8b-71f5af0f530b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:41.524187 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515658dd-af46-4d8b-8a8b-71f5af0f530b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:41.547978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:41.550979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515658dd-af46-4d8b-8a8b-71f5af0f530b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-515658dd-af46-4d8b-8a8b-71f5af0f530b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:41.558562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:41.559461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:41.560245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:41.560846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 190/364] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:41.561462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515658dd-af46-4d8b-8a8b-71f5af0f530b None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:41.568247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6328928-1def-4c55-ba7f-e5f8cab4889a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:41.568901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6328928-1def-4c55-ba7f-e5f8cab4889a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:41.569185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6328928-1def-4c55-ba7f-e5f8cab4889a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:41.569458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6328928-1def-4c55-ba7f-e5f8cab4889a None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:41.570185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6328928-1def-4c55-ba7f-e5f8cab4889a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:41.570485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6328928-1def-4c55-ba7f-e5f8cab4889a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:41.571880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6328928-1def-4c55-ba7f-e5f8cab4889a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:41.584298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 175/365] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:41.585081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6328928-1def-4c55-ba7f-e5f8cab4889a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:41.700341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515658dd-af46-4d8b-8a8b-71f5af0f530b None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:41.709043 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6328928-1def-4c55-ba7f-e5f8cab4889a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:41.745415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de9b43-9173-44a8-8e42-329e9bd80406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:41.746648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de9b43-9173-44a8-8e42-329e9bd80406 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:41.746925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de9b43-9173-44a8-8e42-329e9bd80406 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:41.747194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de9b43-9173-44a8-8e42-329e9bd80406 None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:41.748780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de9b43-9173-44a8-8e42-329e9bd80406 None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:41.750652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de9b43-9173-44a8-8e42-329e9bd80406 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:41.751903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a4eff-a840-48db-8fca-b775fb920aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:41.752679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 191/366] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:41 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:41.752947 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a4eff-a840-48db-8fca-b775fb920aa5 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:41.753450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de9b43-9173-44a8-8e42-329e9bd80406 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:41.753580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a4eff-a840-48db-8fca-b775fb920aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:41.754073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a4eff-a840-48db-8fca-b775fb920aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:41.755005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a4eff-a840-48db-8fca-b775fb920aa5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:41.755508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a4eff-a840-48db-8fca-b775fb920aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:41.757089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a4eff-a840-48db-8fca-b775fb920aa5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:41.769132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 176/367] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:41.770380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a4eff-a840-48db-8fca-b775fb920aa5 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:41.822871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:41.839375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:41.908250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a4eff-a840-48db-8fca-b775fb920aa5 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:41.909657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de9b43-9173-44a8-8e42-329e9bd80406 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:41.951029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce2a86-965d-42e5-9819-4d3313f258ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:41.951754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce2a86-965d-42e5-9819-4d3313f258ae None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:41.952041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce2a86-965d-42e5-9819-4d3313f258ae None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:41.952312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce2a86-965d-42e5-9819-4d3313f258ae None admin] PATH_INFO: `/v3/users/14696bb2b50b43b5b7150ea4b87f5a84` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:41.954375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce2a86-965d-42e5-9819-4d3313f258ae None admin] RBAC: Authorizing `identity:get_user(user_id=14696bb2b50b43b5b7150ea4b87f5a84)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:41.955389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a433791-ddc3-4f8f-8e5b-325da21aa2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:41.955939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce2a86-965d-42e5-9819-4d3313f258ae None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:41.956476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a433791-ddc3-4f8f-8e5b-325da21aa2f4 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:41.956958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a433791-ddc3-4f8f-8e5b-325da21aa2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:41.957439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a433791-ddc3-4f8f-8e5b-325da21aa2f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:41.957874 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 192/368] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:38:41 2022] GET /identity/v3/users/14696bb2b50b43b5b7150ea4b87f5a84 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:41.958878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce2a86-965d-42e5-9819-4d3313f258ae None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:41.976494 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a433791-ddc3-4f8f-8e5b-325da21aa2f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-3a433791-ddc3-4f8f-8e5b-325da21aa2f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:41.985200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:41.986103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:41.986913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:41.990049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 177/369] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:38:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:41.991388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a433791-ddc3-4f8f-8e5b-325da21aa2f4 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:42.101207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce2a86-965d-42e5-9819-4d3313f258ae None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:42.117561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:42.133627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a433791-ddc3-4f8f-8e5b-325da21aa2f4 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:42.140936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b76c95-73e3-49c4-96b0-2f14462463ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:42.141727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b76c95-73e3-49c4-96b0-2f14462463ad None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:42.142157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b76c95-73e3-49c4-96b0-2f14462463ad None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:42.142528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b76c95-73e3-49c4-96b0-2f14462463ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:42.143774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b76c95-73e3-49c4-96b0-2f14462463ad None admin] RBAC: Adding query filter params (group.id=36e9ba2eae5542fd84e9624ca2971649, role.id=47112770ffdc45638da090c45073888e, scope.project.id=805a9d6e523c465393a8f47773ab2b23) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:42.143774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b76c95-73e3-49c4-96b0-2f14462463ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:42.146857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b76c95-73e3-49c4-96b0-2f14462463ad None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:42.154913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 193/370] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 3 14:38:41 2022] GET /identity/v3/role_assignments?group.id=36e9ba2eae5542fd84e9624ca2971649&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=47112770ffdc45638da090c45073888e => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:42.155886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b76c95-73e3-49c4-96b0-2f14462463ad None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:42.156716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:42.187992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbb695-6edb-4a09-8527-5dfc4c44cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:42.188981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbb695-6edb-4a09-8527-5dfc4c44cf19 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:42.189455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbb695-6edb-4a09-8527-5dfc4c44cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:42.190143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbb695-6edb-4a09-8527-5dfc4c44cf19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:42.274644 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbb695-6edb-4a09-8527-5dfc4c44cf19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:42.277010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbb695-6edb-4a09-8527-5dfc4c44cf19 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:42.286534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b76c95-73e3-49c4-96b0-2f14462463ad None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:42.330665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a67b70-8c27-48a4-be81-0a4639359e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:42.331394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a67b70-8c27-48a4-be81-0a4639359e39 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:42.331783 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a67b70-8c27-48a4-be81-0a4639359e39 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:42.332129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a67b70-8c27-48a4-be81-0a4639359e39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:42.333051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a67b70-8c27-48a4-be81-0a4639359e39 None admin] RBAC: Adding query filter params (role.id=8d46737ab3ff453a800fe6b47805eb5a, user.id=14696bb2b50b43b5b7150ea4b87f5a84, scope.system=all) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:42.333432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a67b70-8c27-48a4-be81-0a4639359e39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:42.335215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a67b70-8c27-48a4-be81-0a4639359e39 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:42.349821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 194/371] 10.222.0.46 () {56 vars in 1308 bytes} [Mon Oct 3 14:38:42 2022] GET /identity/v3/role_assignments?user.id=14696bb2b50b43b5b7150ea4b87f5a84&scope.system=all&role.id=8d46737ab3ff453a800fe6b47805eb5a => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:42.350880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a67b70-8c27-48a4-be81-0a4639359e39 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:42.397372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:42.404809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 178/372] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:42.410810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbb695-6edb-4a09-8527-5dfc4c44cf19 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:42.437534 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:42.489531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a67b70-8c27-48a4-be81-0a4639359e39 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:42.535413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba47a58-4c88-4498-bce8-57bb37d22a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:42.536366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba47a58-4c88-4498-bce8-57bb37d22a0c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:42.536998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba47a58-4c88-4498-bce8-57bb37d22a0c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:42.537468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba47a58-4c88-4498-bce8-57bb37d22a0c None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:42.538716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba47a58-4c88-4498-bce8-57bb37d22a0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:42.539214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba47a58-4c88-4498-bce8-57bb37d22a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:42.540825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba47a58-4c88-4498-bce8-57bb37d22a0c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:42.556791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 195/373] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:42.557526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbb695-6edb-4a09-8527-5dfc4c44cf19 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:42.562570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba47a58-4c88-4498-bce8-57bb37d22a0c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:42.607425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367ce48-1349-4bf2-9ceb-f8b9d3b7069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:42.608178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367ce48-1349-4bf2-9ceb-f8b9d3b7069b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:42.608386 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367ce48-1349-4bf2-9ceb-f8b9d3b7069b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:42.608664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367ce48-1349-4bf2-9ceb-f8b9d3b7069b None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:42.609950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367ce48-1349-4bf2-9ceb-f8b9d3b7069b None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:42.611432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367ce48-1349-4bf2-9ceb-f8b9d3b7069b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:42.613448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 179/374] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:42 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:42.619489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367ce48-1349-4bf2-9ceb-f8b9d3b7069b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:42.707578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:42.712256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba47a58-4c88-4498-bce8-57bb37d22a0c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:42.776851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e0f80-d8b1-4e00-b308-33b908f90951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:42.777904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e0f80-d8b1-4e00-b308-33b908f90951 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:42.778397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e0f80-d8b1-4e00-b308-33b908f90951 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:42.778891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e0f80-d8b1-4e00-b308-33b908f90951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:42.811203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:42.840776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367ce48-1349-4bf2-9ceb-f8b9d3b7069b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:42.905040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e0f80-d8b1-4e00-b308-33b908f90951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:42.906921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eac280-082f-46b6-9c3e-2f662609dd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:42.908406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e0f80-d8b1-4e00-b308-33b908f90951 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:42.909293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eac280-082f-46b6-9c3e-2f662609dd79 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:42.909630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eac280-082f-46b6-9c3e-2f662609dd79 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:42.909871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eac280-082f-46b6-9c3e-2f662609dd79 None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:42.912828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eac280-082f-46b6-9c3e-2f662609dd79 None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:42.914297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eac280-082f-46b6-9c3e-2f662609dd79 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:42.916308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 180/375] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:42 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:42.921691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eac280-082f-46b6-9c3e-2f662609dd79 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:43.018796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:43.023970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 196/376] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 14:38:43.031048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e0f80-d8b1-4e00-b308-33b908f90951 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:43.071376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eac280-082f-46b6-9c3e-2f662609dd79 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:43.125432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca7476-2a44-4689-9fa6-b0ef6c03bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:43.126069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca7476-2a44-4689-9fa6-b0ef6c03bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:43.126347 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca7476-2a44-4689-9fa6-b0ef6c03bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:43.126618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca7476-2a44-4689-9fa6-b0ef6c03bbf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:43.127352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca7476-2a44-4689-9fa6-b0ef6c03bbf2 None admin] RBAC: Adding query filter params (group.id=36e9ba2eae5542fd84e9624ca2971649, role.id=8cd21d388193490bac3aa386bcc18763, scope.project.id=805a9d6e523c465393a8f47773ab2b23) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:43.127660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca7476-2a44-4689-9fa6-b0ef6c03bbf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:43.129106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca7476-2a44-4689-9fa6-b0ef6c03bbf2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:43.147936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 181/377] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 3 14:38:42 2022] GET /identity/v3/role_assignments?group.id=36e9ba2eae5542fd84e9624ca2971649&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=8cd21d388193490bac3aa386bcc18763 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:43.173059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e0f80-d8b1-4e00-b308-33b908f90951 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:43.217301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f63c-b802-4c35-8713-aafbecf9bef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:43.217968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f63c-b802-4c35-8713-aafbecf9bef7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:43.218245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f63c-b802-4c35-8713-aafbecf9bef7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:43.218517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f63c-b802-4c35-8713-aafbecf9bef7 None admin] PATH_INFO: `/v3/projects/47484d92809b4b9a846447c512be0a94` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:43.219665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35f63c-b802-4c35-8713-aafbecf9bef7 None admin] RBAC: Authorizing `identity:get_project(project_id=47484d92809b4b9a846447c512be0a94)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:43.221151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35f63c-b802-4c35-8713-aafbecf9bef7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:43.223272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 197/378] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:43 2022] GET /identity/v3/projects/47484d92809b4b9a846447c512be0a94 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:43.229071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca7476-2a44-4689-9fa6-b0ef6c03bbf2 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:43.251359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:43.366934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca7476-2a44-4689-9fa6-b0ef6c03bbf2 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:43.405672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de864e4-0a07-4bd7-9301-a6c66b063a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:43.406330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de864e4-0a07-4bd7-9301-a6c66b063a40 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:43.406654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de864e4-0a07-4bd7-9301-a6c66b063a40 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:43.406887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de864e4-0a07-4bd7-9301-a6c66b063a40 None admin] PATH_INFO: `/v3/groups/8a73142305e440fcaa40de1190133c24` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:43.413974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de864e4-0a07-4bd7-9301-a6c66b063a40 None admin] RBAC: Authorizing `identity:get_group(group_id=8a73142305e440fcaa40de1190133c24)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:43.415512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de864e4-0a07-4bd7-9301-a6c66b063a40 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:43.418072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 182/379] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:43 2022] GET /identity/v3/groups/8a73142305e440fcaa40de1190133c24 => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:43.430711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f63c-b802-4c35-8713-aafbecf9bef7 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:43.513332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:43.552318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f63c-b802-4c35-8713-aafbecf9bef7 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:43.589690 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0986cb5-33e8-4f2c-9210-bbf71d38cc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:43.590305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986cb5-33e8-4f2c-9210-bbf71d38cc86 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:43.590581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986cb5-33e8-4f2c-9210-bbf71d38cc86 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:43.590850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986cb5-33e8-4f2c-9210-bbf71d38cc86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:43.591588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0986cb5-33e8-4f2c-9210-bbf71d38cc86 None admin] RBAC: Adding query filter params (group.id=8a73142305e440fcaa40de1190133c24, role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=47484d92809b4b9a846447c512be0a94) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:43.591894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0986cb5-33e8-4f2c-9210-bbf71d38cc86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:43.593339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0986cb5-33e8-4f2c-9210-bbf71d38cc86 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:43.600200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 198/380] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 3 14:38:43 2022] GET /identity/v3/role_assignments?group.id=8a73142305e440fcaa40de1190133c24&scope.project.id=47484d92809b4b9a846447c512be0a94&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:43.980846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7fac9-acb7-48db-8357-ec0845ca8916 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:43.980846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7fac9-acb7-48db-8357-ec0845ca8916 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:43.980846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7fac9-acb7-48db-8357-ec0845ca8916 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:43.982106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 183/381] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:43.987894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc34b7-722f-4301-bf62-22ce2e262205 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:43.988462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc34b7-722f-4301-bf62-22ce2e262205 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:43.988947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc34b7-722f-4301-bf62-22ce2e262205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:44.009056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d0dc34b7-722f-4301-bf62-22ce2e262205 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:44.011126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d0dc34b7-722f-4301-bf62-22ce2e262205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:44.087204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 199/382] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:44.428766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d43f7c-3ca7-4a90-aca4-87cbf07427cf None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:44.429418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d43f7c-3ca7-4a90-aca4-87cbf07427cf None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:44.429957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d43f7c-3ca7-4a90-aca4-87cbf07427cf None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:44.432323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 184/383] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:44.433563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc34b7-722f-4301-bf62-22ce2e262205 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:44.447742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac78c92-59d6-4c53-886a-fe5f9b008cbb None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:44.448361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac78c92-59d6-4c53-886a-fe5f9b008cbb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:44.448830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac78c92-59d6-4c53-886a-fe5f9b008cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:44.483710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-8ac78c92-59d6-4c53-886a-fe5f9b008cbb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:44.485261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8ac78c92-59d6-4c53-886a-fe5f9b008cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:44.561939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:44.624033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 185/384] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:44.632739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc34b7-722f-4301-bf62-22ce2e262205 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:44.673562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5585d3a-bf09-4b1a-883b-e088e2bdaae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:44.674235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5585d3a-bf09-4b1a-883b-e088e2bdaae0 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:44.674526 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5585d3a-bf09-4b1a-883b-e088e2bdaae0 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:44.674794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5585d3a-bf09-4b1a-883b-e088e2bdaae0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:44.689092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5585d3a-bf09-4b1a-883b-e088e2bdaae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-f5585d3a-bf09-4b1a-883b-e088e2bdaae0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:44.694121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:44.695040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:44.695868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 200/385] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:44.701737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac78c92-59d6-4c53-886a-fe5f9b008cbb None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:44.729915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:44.829985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac78c92-59d6-4c53-886a-fe5f9b008cbb None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:44.875866 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b6309-bd9f-4215-aa7c-6a6f28e01875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:44.876593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b6309-bd9f-4215-aa7c-6a6f28e01875 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:44.876879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b6309-bd9f-4215-aa7c-6a6f28e01875 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:44.877150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b6309-bd9f-4215-aa7c-6a6f28e01875 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:44.877863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b6309-bd9f-4215-aa7c-6a6f28e01875 None admin] RBAC: Adding query filter params (name=member) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:44.878167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b6309-bd9f-4215-aa7c-6a6f28e01875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:44.879542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b6309-bd9f-4215-aa7c-6a6f28e01875 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:44.889010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 186/386] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:44.894706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5585d3a-bf09-4b1a-883b-e088e2bdaae0 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:44.946330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b6309-bd9f-4215-aa7c-6a6f28e01875 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:45.074920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:45.078963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5585d3a-bf09-4b1a-883b-e088e2bdaae0 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:45.116113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b6309-bd9f-4215-aa7c-6a6f28e01875 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:45.117965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:45.156760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e343c-bd8d-458f-afdf-4c353b3b12be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:45.157858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e343c-bd8d-458f-afdf-4c353b3b12be None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:45.158341 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e343c-bd8d-458f-afdf-4c353b3b12be None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:45.158791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e343c-bd8d-458f-afdf-4c353b3b12be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:45.161000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00f4ce-0037-491b-a705-2850ad0ad607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:45.162205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00f4ce-0037-491b-a705-2850ad0ad607 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:45.162205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00f4ce-0037-491b-a705-2850ad0ad607 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:45.162540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00f4ce-0037-491b-a705-2850ad0ad607 None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:45.166530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00f4ce-0037-491b-a705-2850ad0ad607 None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:45.167870 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00f4ce-0037-491b-a705-2850ad0ad607 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:45.169836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 201/387] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:44 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:45.177526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e343c-bd8d-458f-afdf-4c353b3b12be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:45.177670 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00f4ce-0037-491b-a705-2850ad0ad607 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-183e343c-bd8d-458f-afdf-4c353b3b12be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:45.183734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:45.184607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:45.185392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 187/388] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:45.192407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e343c-bd8d-458f-afdf-4c353b3b12be None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:45.314910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e343c-bd8d-458f-afdf-4c353b3b12be None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:45.326618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00f4ce-0037-491b-a705-2850ad0ad607 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:45.346502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:45.352027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1fe03-26cc-4af5-8de0-f56fc1a94157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:45.352765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1fe03-26cc-4af5-8de0-f56fc1a94157 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:45.353293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1fe03-26cc-4af5-8de0-f56fc1a94157 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:45.353765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1fe03-26cc-4af5-8de0-f56fc1a94157 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:45.354611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1fe03-26cc-4af5-8de0-f56fc1a94157 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:45.355007 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1fe03-26cc-4af5-8de0-f56fc1a94157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:45.356531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1fe03-26cc-4af5-8de0-f56fc1a94157 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:45.367367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df9ad6-cb94-421a-bb6c-d1eabb1df274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:45.367932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df9ad6-cb94-421a-bb6c-d1eabb1df274 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:45.368208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df9ad6-cb94-421a-bb6c-d1eabb1df274 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:45.368478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df9ad6-cb94-421a-bb6c-d1eabb1df274 None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:45.369582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 188/389] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:45.370824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458af03-131c-4ee7-9629-72b7478ef126 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:45.371107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458af03-131c-4ee7-9629-72b7478ef126 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:45.371377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458af03-131c-4ee7-9629-72b7478ef126 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:45.372738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 189/390] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:45.373592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458af03-131c-4ee7-9629-72b7478ef126 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:45.374292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df9ad6-cb94-421a-bb6c-d1eabb1df274 None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:45.375697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df9ad6-cb94-421a-bb6c-d1eabb1df274 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:45.377364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 202/391] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:45 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:45.384774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ffe8-fd2b-4bb4-a991-42c42654b6eb None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:45.385054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ffe8-fd2b-4bb4-a991-42c42654b6eb None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:45.385325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ffe8-fd2b-4bb4-a991-42c42654b6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:45.406384 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d3a6ffe8-fd2b-4bb4-a991-42c42654b6eb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:45.407648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d3a6ffe8-fd2b-4bb4-a991-42c42654b6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:45.465750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:45.492431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458af03-131c-4ee7-9629-72b7478ef126 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:45.492831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 203/392] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:45.493978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6ffe8-fd2b-4bb4-a991-42c42654b6eb None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:45.524948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231d9ff-4831-4db7-ab05-8c29d6c69dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:45.525866 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231d9ff-4831-4db7-ab05-8c29d6c69dfe None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:45.526144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231d9ff-4831-4db7-ab05-8c29d6c69dfe None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:45.526415 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231d9ff-4831-4db7-ab05-8c29d6c69dfe None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:45.527885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231d9ff-4831-4db7-ab05-8c29d6c69dfe None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:45.529380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231d9ff-4831-4db7-ab05-8c29d6c69dfe None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:45.531907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 190/393] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:45 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:45.537444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231d9ff-4831-4db7-ab05-8c29d6c69dfe None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:45.607697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:45.649358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231d9ff-4831-4db7-ab05-8c29d6c69dfe None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:45.687984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d843be-2d4d-4f32-8f6b-e5caa802b3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:45.688890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d843be-2d4d-4f32-8f6b-e5caa802b3da None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:45.689298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d843be-2d4d-4f32-8f6b-e5caa802b3da None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:45.689741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d843be-2d4d-4f32-8f6b-e5caa802b3da None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:45.691211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d843be-2d4d-4f32-8f6b-e5caa802b3da None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:45.692963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d843be-2d4d-4f32-8f6b-e5caa802b3da None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:45.693279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6ffe8-fd2b-4bb4-a991-42c42654b6eb None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:45.695700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 191/394] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:45 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:45.702078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d843be-2d4d-4f32-8f6b-e5caa802b3da None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:45.776157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaef6c-c47f-4896-88a6-e0c11682d5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:45.777224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaef6c-c47f-4896-88a6-e0c11682d5fe None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:45.777781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaef6c-c47f-4896-88a6-e0c11682d5fe None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:45.778259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaef6c-c47f-4896-88a6-e0c11682d5fe None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895/groups/36e9ba2eae5542fd84e9624ca2971649/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:45.786840 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aaef6c-c47f-4896-88a6-e0c11682d5fe None admin] RBAC: Authorizing `identity:create_grant(project_id=130f64a6280e4b83bf885edf8986e895, group_id=36e9ba2eae5542fd84e9624ca2971649, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:45.788615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aaef6c-c47f-4896-88a6-e0c11682d5fe None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:45.812289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d843be-2d4d-4f32-8f6b-e5caa802b3da None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:45.823365 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 204/395] 10.222.0.46 () {58 vars in 1442 bytes} [Mon Oct 3 14:38:45 2022] PUT /identity/v3/projects/130f64a6280e4b83bf885edf8986e895/groups/36e9ba2eae5542fd84e9624ca2971649/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:45.846217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:45.849680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883818d4-1a30-4baa-965e-e8bdb1657b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:45.850367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883818d4-1a30-4baa-965e-e8bdb1657b0d None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:45.850650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883818d4-1a30-4baa-965e-e8bdb1657b0d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:45.850928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883818d4-1a30-4baa-965e-e8bdb1657b0d None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895/groups/36e9ba2eae5542fd84e9624ca2971649/roles/8cd21d388193490bac3aa386bcc18763` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:45.853576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883818d4-1a30-4baa-965e-e8bdb1657b0d None admin] RBAC: Authorizing `identity:create_grant(project_id=130f64a6280e4b83bf885edf8986e895, group_id=36e9ba2eae5542fd84e9624ca2971649, role_id=8cd21d388193490bac3aa386bcc18763)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:45.855046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883818d4-1a30-4baa-965e-e8bdb1657b0d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:45.873010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 192/396] 10.222.0.46 () {58 vars in 1442 bytes} [Mon Oct 3 14:38:45 2022] PUT /identity/v3/projects/130f64a6280e4b83bf885edf8986e895/groups/36e9ba2eae5542fd84e9624ca2971649/roles/8cd21d388193490bac3aa386bcc18763 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:38:46.034380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaef6c-c47f-4896-88a6-e0c11682d5fe None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:46.101540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:46.177285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaef6c-c47f-4896-88a6-e0c11682d5fe None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:46.210361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbb254-f1c3-49b0-9646-0c49d67ac15e None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:46.210361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbb254-f1c3-49b0-9646-0c49d67ac15e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:46.210361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbb254-f1c3-49b0-9646-0c49d67ac15e None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:46.211831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 193/397] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:46.217312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513dbf3-5b10-4400-8fc1-8be0b4faa5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:46.218289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513dbf3-5b10-4400-8fc1-8be0b4faa5c7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:46.218394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299571-c5a9-45d4-8e98-d8ea6f877f5f None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:46.218591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299571-c5a9-45d4-8e98-d8ea6f877f5f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:46.218870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299571-c5a9-45d4-8e98-d8ea6f877f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:46.219251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513dbf3-5b10-4400-8fc1-8be0b4faa5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:46.219723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513dbf3-5b10-4400-8fc1-8be0b4faa5c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:46.232067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513dbf3-5b10-4400-8fc1-8be0b4faa5c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-9513dbf3-5b10-4400-8fc1-8be0b4faa5c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:46.237721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:46.238581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:46.239437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:46.240014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-26299571-c5a9-45d4-8e98-d8ea6f877f5f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:46.241189 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-26299571-c5a9-45d4-8e98-d8ea6f877f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:46.242281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 205/398] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:46.248295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513dbf3-5b10-4400-8fc1-8be0b4faa5c7 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:46.335270 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 194/399] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:46.338949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83619af4-6097-4752-881f-c0335b08f556 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:46.339453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83619af4-6097-4752-881f-c0335b08f556 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:46.339939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83619af4-6097-4752-881f-c0335b08f556 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:46.341452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 195/400] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:46.346903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5944-64f7-46aa-b74f-31be99f799df None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:46.348229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5944-64f7-46aa-b74f-31be99f799df None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:46.349424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5944-64f7-46aa-b74f-31be99f799df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:46.389020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513dbf3-5b10-4400-8fc1-8be0b4faa5c7 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:46.392441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-fafc5944-64f7-46aa-b74f-31be99f799df None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:46.399192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-fafc5944-64f7-46aa-b74f-31be99f799df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:46.403053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:46.455319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e683d27-b09a-4395-971b-8033bcc71ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:46.456285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e683d27-b09a-4395-971b-8033bcc71ded None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:46.456573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e683d27-b09a-4395-971b-8033bcc71ded None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:46.456845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e683d27-b09a-4395-971b-8033bcc71ded None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:46.457664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e683d27-b09a-4395-971b-8033bcc71ded None admin] RBAC: Adding query filter params (name=member) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:46.458026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e683d27-b09a-4395-971b-8033bcc71ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:46.459615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e683d27-b09a-4395-971b-8033bcc71ded None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:46.480568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 206/401] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:46.499029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e683d27-b09a-4395-971b-8033bcc71ded None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:46.511001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 196/402] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:46.606461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e683d27-b09a-4395-971b-8033bcc71ded None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:46.640860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67d471-9d4d-4893-9414-18f211586ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:46.642045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67d471-9d4d-4893-9414-18f211586ea7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:46.642516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67d471-9d4d-4893-9414-18f211586ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:46.642973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67d471-9d4d-4893-9414-18f211586ea7 None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:46.643073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc5944-64f7-46aa-b74f-31be99f799df None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:46.644366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67d471-9d4d-4893-9414-18f211586ea7 None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:46.647379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67d471-9d4d-4893-9414-18f211586ea7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:46.653901 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 207/403] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:46 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:46.656532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67d471-9d4d-4893-9414-18f211586ea7 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:46.699988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:46.705851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:46.767056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc5944-64f7-46aa-b74f-31be99f799df None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:46.803584 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0a3ce-05ba-406d-b758-ea3180bec10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:46.804600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0a3ce-05ba-406d-b758-ea3180bec10b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:46.805074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0a3ce-05ba-406d-b758-ea3180bec10b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:46.805565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0a3ce-05ba-406d-b758-ea3180bec10b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:46.817380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0a3ce-05ba-406d-b758-ea3180bec10b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-60d0a3ce-05ba-406d-b758-ea3180bec10b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:46.822690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:46.823527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:46.824294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:46.826843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 197/404] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:46.832071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0a3ce-05ba-406d-b758-ea3180bec10b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:46.863568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67d471-9d4d-4893-9414-18f211586ea7 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:46.905845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e959d74-9c58-4e46-a3b7-a3c1b391c7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:46.906479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e959d74-9c58-4e46-a3b7-a3c1b391c7b2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:46.906754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e959d74-9c58-4e46-a3b7-a3c1b391c7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:46.907022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e959d74-9c58-4e46-a3b7-a3c1b391c7b2 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:46.908154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e959d74-9c58-4e46-a3b7-a3c1b391c7b2 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:46.910252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e959d74-9c58-4e46-a3b7-a3c1b391c7b2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:46.916893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 208/405] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:46 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:46.916893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e959d74-9c58-4e46-a3b7-a3c1b391c7b2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:46.947507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0a3ce-05ba-406d-b758-ea3180bec10b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:46.959411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:46.987552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ab327-4169-4fec-a00a-b5c5fc366bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:46.988573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ab327-4169-4fec-a00a-b5c5fc366bef None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:46.989061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ab327-4169-4fec-a00a-b5c5fc366bef None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:46.989576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ab327-4169-4fec-a00a-b5c5fc366bef None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:46.991169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ab327-4169-4fec-a00a-b5c5fc366bef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:46.992237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ab327-4169-4fec-a00a-b5c5fc366bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:46.994451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ab327-4169-4fec-a00a-b5c5fc366bef None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:47.008627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 198/406] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:47.011054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ab327-4169-4fec-a00a-b5c5fc366bef None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:47.035185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e959d74-9c58-4e46-a3b7-a3c1b391c7b2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:47.038822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:47.082505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574519e-edd5-43f4-b0a2-0e9671d3b610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:47.083434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574519e-edd5-43f4-b0a2-0e9671d3b610 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:47.083718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574519e-edd5-43f4-b0a2-0e9671d3b610 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:47.083991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574519e-edd5-43f4-b0a2-0e9671d3b610 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:47.098534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574519e-edd5-43f4-b0a2-0e9671d3b610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-4574519e-edd5-43f4-b0a2-0e9671d3b610 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:47.105478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:47.106367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 209/407] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:38:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:47.107752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574519e-edd5-43f4-b0a2-0e9671d3b610 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:47.139253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ab327-4169-4fec-a00a-b5c5fc366bef None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:47.179977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40126f77-7f1b-4a6a-a366-9c6685ab940e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:47.181198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40126f77-7f1b-4a6a-a366-9c6685ab940e None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:47.181717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40126f77-7f1b-4a6a-a366-9c6685ab940e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:47.182637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40126f77-7f1b-4a6a-a366-9c6685ab940e None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23/groups/36e9ba2eae5542fd84e9624ca2971649/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:47.185199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40126f77-7f1b-4a6a-a366-9c6685ab940e None admin] RBAC: Authorizing `identity:create_grant(project_id=805a9d6e523c465393a8f47773ab2b23, group_id=36e9ba2eae5542fd84e9624ca2971649, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:47.186863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40126f77-7f1b-4a6a-a366-9c6685ab940e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:47.209787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 199/408] 10.222.0.46 () {58 vars in 1442 bytes} [Mon Oct 3 14:38:47 2022] PUT /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23/groups/36e9ba2eae5542fd84e9624ca2971649/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:47.209787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40126f77-7f1b-4a6a-a366-9c6685ab940e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:47.264738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574519e-edd5-43f4-b0a2-0e9671d3b610 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:47.280133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:47.323555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbadd1-a423-4831-9c0f-fa48c3496a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:47.324204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbadd1-a423-4831-9c0f-fa48c3496a6b None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:47.324479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbadd1-a423-4831-9c0f-fa48c3496a6b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:47.324748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbadd1-a423-4831-9c0f-fa48c3496a6b None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:47.326050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbadd1-a423-4831-9c0f-fa48c3496a6b None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:47.327955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbadd1-a423-4831-9c0f-fa48c3496a6b None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:47.330534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 210/409] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:47 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:47.336639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbadd1-a423-4831-9c0f-fa48c3496a6b None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:47.378836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:47.432830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40126f77-7f1b-4a6a-a366-9c6685ab940e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:47.455930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbadd1-a423-4831-9c0f-fa48c3496a6b None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:47.473087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e420-4efe-410b-a961-b0f1bcef484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:47.473743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e420-4efe-410b-a961-b0f1bcef484d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:47.474023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e420-4efe-410b-a961-b0f1bcef484d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:47.474295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e420-4efe-410b-a961-b0f1bcef484d None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:47.475247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24e420-4efe-410b-a961-b0f1bcef484d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:47.475554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24e420-4efe-410b-a961-b0f1bcef484d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:47.476920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24e420-4efe-410b-a961-b0f1bcef484d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:47.488164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 200/410] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:47.496528 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e420-4efe-410b-a961-b0f1bcef484d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:47.500278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532b082-1db7-4c65-9c69-2333bd07afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:47.501138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532b082-1db7-4c65-9c69-2333bd07afd1 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:47.501640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532b082-1db7-4c65-9c69-2333bd07afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:47.502170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532b082-1db7-4c65-9c69-2333bd07afd1 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:47.503473 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532b082-1db7-4c65-9c69-2333bd07afd1 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:47.505205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532b082-1db7-4c65-9c69-2333bd07afd1 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:47.507459 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 211/411] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:47 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:47.520536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532b082-1db7-4c65-9c69-2333bd07afd1 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:47.606953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:47.616924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e420-4efe-410b-a961-b0f1bcef484d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:47.691591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532b082-1db7-4c65-9c69-2333bd07afd1 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:47.701852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:47.714190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1e3ad-27a4-4cb6-b6d6-1dbe9cbb42e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:47.714842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1e3ad-27a4-4cb6-b6d6-1dbe9cbb42e9 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:47.715128 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1e3ad-27a4-4cb6-b6d6-1dbe9cbb42e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:47.715403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1e3ad-27a4-4cb6-b6d6-1dbe9cbb42e9 None admin] PATH_INFO: `/v3/groups/8a73142305e440fcaa40de1190133c24` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:47.722183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1e3ad-27a4-4cb6-b6d6-1dbe9cbb42e9 None admin] RBAC: Authorizing `identity:get_group(group_id=8a73142305e440fcaa40de1190133c24)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:47.723628 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1e3ad-27a4-4cb6-b6d6-1dbe9cbb42e9 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:47.725561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 201/412] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:47 2022] GET /identity/v3/groups/8a73142305e440fcaa40de1190133c24 => generated 229 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:47.727373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e6058-4775-4251-8c53-539c3e1608cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:47.729148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6058-4775-4251-8c53-539c3e1608cc None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:47.729674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6058-4775-4251-8c53-539c3e1608cc None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:47.730170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6058-4775-4251-8c53-539c3e1608cc None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23/groups/36e9ba2eae5542fd84e9624ca2971649/roles/8cd21d388193490bac3aa386bcc18763` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:47.731386 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1e3ad-27a4-4cb6-b6d6-1dbe9cbb42e9 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:47.732838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e6058-4775-4251-8c53-539c3e1608cc None admin] RBAC: Authorizing `identity:create_grant(project_id=805a9d6e523c465393a8f47773ab2b23, group_id=36e9ba2eae5542fd84e9624ca2971649, role_id=8cd21d388193490bac3aa386bcc18763)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:47.734506 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e6058-4775-4251-8c53-539c3e1608cc None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:47.753678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 212/413] 10.222.0.46 () {58 vars in 1442 bytes} [Mon Oct 3 14:38:47 2022] PUT /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23/groups/36e9ba2eae5542fd84e9624ca2971649/roles/8cd21d388193490bac3aa386bcc18763 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:47.862511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1e3ad-27a4-4cb6-b6d6-1dbe9cbb42e9 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:47.900993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9ba18-fb67-4598-b2ce-06c813817970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:47.902094 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9ba18-fb67-4598-b2ce-06c813817970 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:47.902634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9ba18-fb67-4598-b2ce-06c813817970 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:47.903101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9ba18-fb67-4598-b2ce-06c813817970 None admin] PATH_INFO: `/v3/projects/47484d92809b4b9a846447c512be0a94` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:47.904588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b9ba18-fb67-4598-b2ce-06c813817970 None admin] RBAC: Authorizing `identity:get_project(project_id=47484d92809b4b9a846447c512be0a94)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:47.906339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b9ba18-fb67-4598-b2ce-06c813817970 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:47.908577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 202/414] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:47 2022] GET /identity/v3/projects/47484d92809b4b9a846447c512be0a94 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:47.914259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e6058-4775-4251-8c53-539c3e1608cc None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:48.043876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e6058-4775-4251-8c53-539c3e1608cc None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:48.093134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa87531-5164-4f5e-bf8a-e19bb1892a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:48.094273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa87531-5164-4f5e-bf8a-e19bb1892a58 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:48.094757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa87531-5164-4f5e-bf8a-e19bb1892a58 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:48.095227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa87531-5164-4f5e-bf8a-e19bb1892a58 None admin] PATH_INFO: `/v3/projects/47484d92809b4b9a846447c512be0a94/groups/8a73142305e440fcaa40de1190133c24/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:48.097811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa87531-5164-4f5e-bf8a-e19bb1892a58 None admin] RBAC: Authorizing `identity:create_grant(project_id=47484d92809b4b9a846447c512be0a94, group_id=8a73142305e440fcaa40de1190133c24, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:48.099535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa87531-5164-4f5e-bf8a-e19bb1892a58 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:48.124898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 213/415] 10.222.0.46 () {58 vars in 1442 bytes} [Mon Oct 3 14:38:47 2022] PUT /identity/v3/projects/47484d92809b4b9a846447c512be0a94/groups/8a73142305e440fcaa40de1190133c24/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:38:48.499794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28be38-29a5-495d-8d25-8742ca95ef48 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:48.500821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28be38-29a5-495d-8d25-8742ca95ef48 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:48.501329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28be38-29a5-495d-8d25-8742ca95ef48 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:48.503188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 203/416] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:48.508416 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c116ec8-e3e1-4b40-9bb9-2563e223530f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:48.508700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c116ec8-e3e1-4b40-9bb9-2563e223530f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:48.509526 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c116ec8-e3e1-4b40-9bb9-2563e223530f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:48.538662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-3c116ec8-e3e1-4b40-9bb9-2563e223530f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:48.539850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3c116ec8-e3e1-4b40-9bb9-2563e223530f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:48.549466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbada2-fe5e-4566-a3f7-3a52161ea592 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:48.550037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbada2-fe5e-4566-a3f7-3a52161ea592 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:48.550526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbada2-fe5e-4566-a3f7-3a52161ea592 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:48.552328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 204/417] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:48.557242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fb208-c91b-4628-9b4c-35db6958ffe1 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:48.557514 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fb208-c91b-4628-9b4c-35db6958ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:48.557829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fb208-c91b-4628-9b4c-35db6958ffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:48.560105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:48.594665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-3f7fb208-c91b-4628-9b4c-35db6958ffe1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:48.595865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3f7fb208-c91b-4628-9b4c-35db6958ffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:48.640339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 214/418] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:48.645202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:48.693073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 205/419] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:48.935793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c116ec8-e3e1-4b40-9bb9-2563e223530f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:49.060739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c116ec8-e3e1-4b40-9bb9-2563e223530f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:49.092876 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fb208-c91b-4628-9b4c-35db6958ffe1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:49.125534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f6f9e-5126-44c4-b8d1-3cac099941b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:49.126514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f6f9e-5126-44c4-b8d1-3cac099941b5 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:49.127013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f6f9e-5126-44c4-b8d1-3cac099941b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:49.127488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f6f9e-5126-44c4-b8d1-3cac099941b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:49.151694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f6f9e-5126-44c4-b8d1-3cac099941b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-be0f6f9e-5126-44c4-b8d1-3cac099941b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:49.156981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:49.157892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:49.158779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:49.167379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 215/420] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:49.173334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f6f9e-5126-44c4-b8d1-3cac099941b5 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:49.199721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:49.249532 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fb208-c91b-4628-9b4c-35db6958ffe1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:49.298230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61b797-b063-40c7-be32-9fbaf44822c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:49.299110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61b797-b063-40c7-be32-9fbaf44822c0 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:49.299582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61b797-b063-40c7-be32-9fbaf44822c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:49.300069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61b797-b063-40c7-be32-9fbaf44822c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:49.330538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61b797-b063-40c7-be32-9fbaf44822c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-6d61b797-b063-40c7-be32-9fbaf44822c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:49.338622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:49.339511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:49.340291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:49.343308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 206/421] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:49.349577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61b797-b063-40c7-be32-9fbaf44822c0 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:49.368214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f6f9e-5126-44c4-b8d1-3cac099941b5 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:49.374537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:49.438190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdba4e5-626b-42b5-9d95-28771b7a9463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:49.438793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdba4e5-626b-42b5-9d95-28771b7a9463 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:49.439075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdba4e5-626b-42b5-9d95-28771b7a9463 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:49.439342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdba4e5-626b-42b5-9d95-28771b7a9463 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:49.440062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdba4e5-626b-42b5-9d95-28771b7a9463 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:49.440371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdba4e5-626b-42b5-9d95-28771b7a9463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:49.441750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdba4e5-626b-42b5-9d95-28771b7a9463 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:49.454744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 216/422] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:49.460270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdba4e5-626b-42b5-9d95-28771b7a9463 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:49.460565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61b797-b063-40c7-be32-9fbaf44822c0 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:49.502903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b07f8d-5b62-465c-a234-45b0e02d7c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:49.504329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b07f8d-5b62-465c-a234-45b0e02d7c85 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:49.505171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b07f8d-5b62-465c-a234-45b0e02d7c85 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:49.505860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b07f8d-5b62-465c-a234-45b0e02d7c85 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:49.507842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b07f8d-5b62-465c-a234-45b0e02d7c85 None admin] RBAC: Adding query filter params (name=member) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:49.509663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b07f8d-5b62-465c-a234-45b0e02d7c85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:49.511262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b07f8d-5b62-465c-a234-45b0e02d7c85 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:49.528631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 207/423] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:49.534258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b07f8d-5b62-465c-a234-45b0e02d7c85 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:49.605857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:49.639961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:49.663263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdba4e5-626b-42b5-9d95-28771b7a9463 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:49.664081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b07f8d-5b62-465c-a234-45b0e02d7c85 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:49.702817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7160fb5-cb73-4103-8fe1-a4a6d1940ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:49.703831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7160fb5-cb73-4103-8fe1-a4a6d1940ca1 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:49.704316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7160fb5-cb73-4103-8fe1-a4a6d1940ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:49.704878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7160fb5-cb73-4103-8fe1-a4a6d1940ca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:49.753673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e25bd3-c417-4ddd-a8fb-3aa3cf1541cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:49.754288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e25bd3-c417-4ddd-a8fb-3aa3cf1541cd None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:49.754563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e25bd3-c417-4ddd-a8fb-3aa3cf1541cd None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:49.754825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e25bd3-c417-4ddd-a8fb-3aa3cf1541cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:49.789326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7160fb5-cb73-4103-8fe1-a4a6d1940ca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:49.791062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7160fb5-cb73-4103-8fe1-a4a6d1940ca1 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:49.852775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e25bd3-c417-4ddd-a8fb-3aa3cf1541cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:49.855080 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e25bd3-c417-4ddd-a8fb-3aa3cf1541cd None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:49.877284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:49.905737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 208/424] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:49.907222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71828-8f4a-4a64-acaa-5715ef6e72eb None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:49.907684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71828-8f4a-4a64-acaa-5715ef6e72eb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:49.908612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71828-8f4a-4a64-acaa-5715ef6e72eb None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:49.910594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 209/425] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:49.911457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d71828-8f4a-4a64-acaa-5715ef6e72eb None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:49.950439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:49.992119 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 217/426] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 14:38:49.994362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9673a-2ab4-4665-9449-e4b80b9d8666 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:49.994848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9673a-2ab4-4665-9449-e4b80b9d8666 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:49.995673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9673a-2ab4-4665-9449-e4b80b9d8666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.016145 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d4f9673a-2ab4-4665-9449-e4b80b9d8666 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:50.017469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d4f9673a-2ab4-4665-9449-e4b80b9d8666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:50.035661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d71828-8f4a-4a64-acaa-5715ef6e72eb None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:50.075046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0387c2-80ef-4410-a5dd-10460c97185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:50.076146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0387c2-80ef-4410-a5dd-10460c97185f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.076604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0387c2-80ef-4410-a5dd-10460c97185f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.077374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0387c2-80ef-4410-a5dd-10460c97185f None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.079014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0387c2-80ef-4410-a5dd-10460c97185f None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:50.080926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0387c2-80ef-4410-a5dd-10460c97185f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:50.083548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 210/427] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:49 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:50.084934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0387c2-80ef-4410-a5dd-10460c97185f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:50.093827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 218/428] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:50.094605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9673a-2ab4-4665-9449-e4b80b9d8666 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:50.139397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:50.195217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0387c2-80ef-4410-a5dd-10460c97185f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:50.236407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7780e9c-d91a-4756-9e82-66caec1f212a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:50.237656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7780e9c-d91a-4756-9e82-66caec1f212a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.238117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7780e9c-d91a-4756-9e82-66caec1f212a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.238593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7780e9c-d91a-4756-9e82-66caec1f212a None admin] PATH_INFO: `/v3/projects/130f64a6280e4b83bf885edf8986e895` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.239924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7780e9c-d91a-4756-9e82-66caec1f212a None admin] RBAC: Authorizing `identity:get_project(project_id=130f64a6280e4b83bf885edf8986e895)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:50.242617 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7780e9c-d91a-4756-9e82-66caec1f212a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:50.245418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 211/429] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:50 2022] GET /identity/v3/projects/130f64a6280e4b83bf885edf8986e895 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:50.250460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7780e9c-d91a-4756-9e82-66caec1f212a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:50.257767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9673a-2ab4-4665-9449-e4b80b9d8666 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:50.259916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:50.305468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8b2d-2672-4a11-b459-47673abe6ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:50.306439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8b2d-2672-4a11-b459-47673abe6ef4 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.306919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8b2d-2672-4a11-b459-47673abe6ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.307379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8b2d-2672-4a11-b459-47673abe6ef4 None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.308680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d8b2d-2672-4a11-b459-47673abe6ef4 None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:50.310333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d8b2d-2672-4a11-b459-47673abe6ef4 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:50.312447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 219/430] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:50 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:50.318264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8b2d-2672-4a11-b459-47673abe6ef4 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:50.421362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7780e9c-d91a-4756-9e82-66caec1f212a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:50.466242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:50.485675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444290e-02fb-48d2-9a09-237541983b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:50.486385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444290e-02fb-48d2-9a09-237541983b5a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.486664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444290e-02fb-48d2-9a09-237541983b5a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.486932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444290e-02fb-48d2-9a09-237541983b5a None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.490509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444290e-02fb-48d2-9a09-237541983b5a None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:50.491954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444290e-02fb-48d2-9a09-237541983b5a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:50.497381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 212/431] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:50 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:50.500557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444290e-02fb-48d2-9a09-237541983b5a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:50.524626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8b2d-2672-4a11-b459-47673abe6ef4 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:50.563601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f443-1f45-4f41-ae33-815b05690f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:50.564596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f443-1f45-4f41-ae33-815b05690f83 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.564937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f443-1f45-4f41-ae33-815b05690f83 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.565281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f443-1f45-4f41-ae33-815b05690f83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.566165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8f443-1f45-4f41-ae33-815b05690f83 None admin] RBAC: Adding query filter params (group.id=36e9ba2eae5542fd84e9624ca2971649, role.id=47112770ffdc45638da090c45073888e, scope.project.id=130f64a6280e4b83bf885edf8986e895) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:50.566560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8f443-1f45-4f41-ae33-815b05690f83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:50.568088 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8f443-1f45-4f41-ae33-815b05690f83 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:50.575535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 220/432] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 3 14:38:50 2022] GET /identity/v3/role_assignments?group.id=36e9ba2eae5542fd84e9624ca2971649&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=47112770ffdc45638da090c45073888e => generated 611 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:50.582531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f443-1f45-4f41-ae33-815b05690f83 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:50.625885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444290e-02fb-48d2-9a09-237541983b5a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:50.671450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b9f70-38ca-4e7c-ad4d-63b89c48b17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:50.672633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b9f70-38ca-4e7c-ad4d-63b89c48b17b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.673083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b9f70-38ca-4e7c-ad4d-63b89c48b17b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.673560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b9f70-38ca-4e7c-ad4d-63b89c48b17b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.674535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:50.676435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b9f70-38ca-4e7c-ad4d-63b89c48b17b None admin] RBAC: Adding query filter params (group.id=36e9ba2eae5542fd84e9624ca2971649, role.id=8cd21d388193490bac3aa386bcc18763, scope.project.id=130f64a6280e4b83bf885edf8986e895) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:50.676950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b9f70-38ca-4e7c-ad4d-63b89c48b17b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:50.678685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b9f70-38ca-4e7c-ad4d-63b89c48b17b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:50.687746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 213/433] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 3 14:38:50 2022] GET /identity/v3/role_assignments?group.id=36e9ba2eae5542fd84e9624ca2971649&scope.project.id=130f64a6280e4b83bf885edf8986e895&role.id=8cd21d388193490bac3aa386bcc18763 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:50.689915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2f6e-b428-468e-84b0-dacbcb92e5cd None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.690202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2f6e-b428-468e-84b0-dacbcb92e5cd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.690503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2f6e-b428-468e-84b0-dacbcb92e5cd None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.691820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 214/434] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:50.698477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bb9ea-b8d0-4a2e-b259-0a0dcfa74dce None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.698814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bb9ea-b8d0-4a2e-b259-0a0dcfa74dce None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.699027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bb9ea-b8d0-4a2e-b259-0a0dcfa74dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.719377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-832bb9ea-b8d0-4a2e-b259-0a0dcfa74dce None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:50.721194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-832bb9ea-b8d0-4a2e-b259-0a0dcfa74dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:50.768897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f443-1f45-4f41-ae33-815b05690f83 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:50.785912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:50.811318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37d6f1-ed83-4337-acf2-2a619127bb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:50.812137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37d6f1-ed83-4337-acf2-2a619127bb19 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.812478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37d6f1-ed83-4337-acf2-2a619127bb19 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.812864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37d6f1-ed83-4337-acf2-2a619127bb19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.817318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 215/435] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:50.819073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81052e-1270-4a92-b452-b9e8f5d47b42 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.819541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81052e-1270-4a92-b452-b9e8f5d47b42 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.819967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81052e-1270-4a92-b452-b9e8f5d47b42 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.822026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 216/436] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:50.828987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e8354-58ff-47f1-b2c0-ca7f78099e25 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:50.829819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e8354-58ff-47f1-b2c0-ca7f78099e25 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:50.830478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e8354-58ff-47f1-b2c0-ca7f78099e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:50.836526 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37d6f1-ed83-4337-acf2-2a619127bb19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-6e37d6f1-ed83-4337-acf2-2a619127bb19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:50.842819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:50.843771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:50.844536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 221/437] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:50.849949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37d6f1-ed83-4337-acf2-2a619127bb19 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:50.855967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-0a8e8354-58ff-47f1-b2c0-ca7f78099e25 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:50.857190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0a8e8354-58ff-47f1-b2c0-ca7f78099e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:50.944303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 217/438] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:50.967960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37d6f1-ed83-4337-acf2-2a619127bb19 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:50.969733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:51.005032 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54069ae-ce40-4a9e-adb8-169508564b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:51.005669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54069ae-ce40-4a9e-adb8-169508564b8a None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:51.005949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54069ae-ce40-4a9e-adb8-169508564b8a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:51.006220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54069ae-ce40-4a9e-adb8-169508564b8a None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:51.006889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54069ae-ce40-4a9e-adb8-169508564b8a None admin] RBAC: Adding query filter params (name=member) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:51.007190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54069ae-ce40-4a9e-adb8-169508564b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:51.008575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54069ae-ce40-4a9e-adb8-169508564b8a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:51.018055 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 222/439] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:38:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:51.023430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e8354-58ff-47f1-b2c0-ca7f78099e25 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:51.132971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e8354-58ff-47f1-b2c0-ca7f78099e25 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:51.135173 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54069ae-ce40-4a9e-adb8-169508564b8a None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:51.145056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:51.175231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6658d-e885-4053-bbcb-c45fc7241a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:51.175855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6658d-e885-4053-bbcb-c45fc7241a4e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:51.176134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6658d-e885-4053-bbcb-c45fc7241a4e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:51.176407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6658d-e885-4053-bbcb-c45fc7241a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:51.258385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6658d-e885-4053-bbcb-c45fc7241a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:51.260413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6658d-e885-4053-bbcb-c45fc7241a4e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:51.271576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54069ae-ce40-4a9e-adb8-169508564b8a None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:51.316688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88328d-c808-4375-97b9-82d7430b6b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:51.317541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88328d-c808-4375-97b9-82d7430b6b1d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:51.317648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88328d-c808-4375-97b9-82d7430b6b1d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:51.317919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88328d-c808-4375-97b9-82d7430b6b1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:51.330452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88328d-c808-4375-97b9-82d7430b6b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-ae88328d-c808-4375-97b9-82d7430b6b1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:51.335873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:51.336728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:51.337583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 223/440] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:38:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:51.338885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88328d-c808-4375-97b9-82d7430b6b1d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:51.360809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:51.390042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 218/441] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:51.390886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6658d-e885-4053-bbcb-c45fc7241a4e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:51.404092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:51.447945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88328d-c808-4375-97b9-82d7430b6b1d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:51.480416 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f875d4-7c20-4512-b553-5990337f7f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:51.481028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f875d4-7c20-4512-b553-5990337f7f3f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:51.481303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f875d4-7c20-4512-b553-5990337f7f3f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:51.481601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f875d4-7c20-4512-b553-5990337f7f3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:51.489704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6658d-e885-4053-bbcb-c45fc7241a4e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:51.491382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f875d4-7c20-4512-b553-5990337f7f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-35f875d4-7c20-4512-b553-5990337f7f3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:51.496158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:51.497138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:51.497979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 224/442] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:51.498666 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f875d4-7c20-4512-b553-5990337f7f3f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:51.529650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e7963-36a1-4543-8fcb-76c00c6614c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:51.530250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e7963-36a1-4543-8fcb-76c00c6614c6 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:51.530529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e7963-36a1-4543-8fcb-76c00c6614c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:51.530800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e7963-36a1-4543-8fcb-76c00c6614c6 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:51.531466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e7963-36a1-4543-8fcb-76c00c6614c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:51.531776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e7963-36a1-4543-8fcb-76c00c6614c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:51.533153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e7963-36a1-4543-8fcb-76c00c6614c6 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:51.541475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 219/443] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:38:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:51.542273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e7963-36a1-4543-8fcb-76c00c6614c6 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:51.576096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:51.598017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f875d4-7c20-4512-b553-5990337f7f3f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:51.609223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:51.632363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd938bc-2b05-4bdb-91fd-9cf60712bc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:51.632983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd938bc-2b05-4bdb-91fd-9cf60712bc01 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:51.633263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd938bc-2b05-4bdb-91fd-9cf60712bc01 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:51.633556 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd938bc-2b05-4bdb-91fd-9cf60712bc01 None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:51.634695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd938bc-2b05-4bdb-91fd-9cf60712bc01 None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:51.636146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd938bc-2b05-4bdb-91fd-9cf60712bc01 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:51.637786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 225/444] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:51 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:51.638550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd938bc-2b05-4bdb-91fd-9cf60712bc01 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:51.639405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e7963-36a1-4543-8fcb-76c00c6614c6 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:51.673762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b20ff-3325-43c1-8488-b3ad7761dd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:51.674357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b20ff-3325-43c1-8488-b3ad7761dd8f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:51.674633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b20ff-3325-43c1-8488-b3ad7761dd8f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:51.674902 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b20ff-3325-43c1-8488-b3ad7761dd8f None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:51.675554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b20ff-3325-43c1-8488-b3ad7761dd8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:51.675855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b20ff-3325-43c1-8488-b3ad7761dd8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:51.677208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b20ff-3325-43c1-8488-b3ad7761dd8f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:51.684793 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 220/445] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:51.685665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b20ff-3325-43c1-8488-b3ad7761dd8f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:51.741788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd938bc-2b05-4bdb-91fd-9cf60712bc01 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:51.774354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3724a-8622-42fe-aee5-afc294b03055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:51.774999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3724a-8622-42fe-aee5-afc294b03055 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:51.775274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3724a-8622-42fe-aee5-afc294b03055 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:51.775544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3724a-8622-42fe-aee5-afc294b03055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:51.782926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b20ff-3325-43c1-8488-b3ad7761dd8f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:51.789697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:51.815735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312e4ad-3bef-4d4d-839c-193900dee479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:51.816356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312e4ad-3bef-4d4d-839c-193900dee479 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:51.816637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312e4ad-3bef-4d4d-839c-193900dee479 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:51.816909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312e4ad-3bef-4d4d-839c-193900dee479 None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:51.818041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312e4ad-3bef-4d4d-839c-193900dee479 None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:51.819465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312e4ad-3bef-4d4d-839c-193900dee479 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:51.821027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 221/446] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:51 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:51.821809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312e4ad-3bef-4d4d-839c-193900dee479 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:51.824166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:51.842572 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3724a-8622-42fe-aee5-afc294b03055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:51.844009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3724a-8622-42fe-aee5-afc294b03055 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:51.920346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312e4ad-3bef-4d4d-839c-193900dee479 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:51.939538 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 226/447] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:51.940259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3724a-8622-42fe-aee5-afc294b03055 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:51.953536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0afac-0265-4e2f-a1a0-a40cbda6ee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:51.954139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0afac-0265-4e2f-a1a0-a40cbda6ee6d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:51.954412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0afac-0265-4e2f-a1a0-a40cbda6ee6d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:51.954679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0afac-0265-4e2f-a1a0-a40cbda6ee6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:51.986067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:52.021865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0afac-0265-4e2f-a1a0-a40cbda6ee6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:52.023323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0afac-0265-4e2f-a1a0-a40cbda6ee6d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:52.025651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:52.041160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3724a-8622-42fe-aee5-afc294b03055 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:52.073804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b3c0f-4ec4-463e-9f52-37e43c0fdbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:52.074395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b3c0f-4ec4-463e-9f52-37e43c0fdbba None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:52.074671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b3c0f-4ec4-463e-9f52-37e43c0fdbba None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:52.074934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b3c0f-4ec4-463e-9f52-37e43c0fdbba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:52.075652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b3c0f-4ec4-463e-9f52-37e43c0fdbba None admin] RBAC: Adding query filter params (group.id=36e9ba2eae5542fd84e9624ca2971649, role.id=47112770ffdc45638da090c45073888e, scope.project.id=805a9d6e523c465393a8f47773ab2b23) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:52.075957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b3c0f-4ec4-463e-9f52-37e43c0fdbba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:52.077372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b3c0f-4ec4-463e-9f52-37e43c0fdbba None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:52.083480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 227/448] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 3 14:38:51 2022] GET /identity/v3/role_assignments?group.id=36e9ba2eae5542fd84e9624ca2971649&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=47112770ffdc45638da090c45073888e => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:52.084238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b3c0f-4ec4-463e-9f52-37e43c0fdbba None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:52.125239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 222/449] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:38:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:38:52.130720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0afac-0265-4e2f-a1a0-a40cbda6ee6d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:52.204618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b3c0f-4ec4-463e-9f52-37e43c0fdbba None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:52.214607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:52.240960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f10f5-9039-4161-8ea0-d3d32f1cba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:52.241632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f10f5-9039-4161-8ea0-d3d32f1cba83 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:52.241910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f10f5-9039-4161-8ea0-d3d32f1cba83 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:52.242182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f10f5-9039-4161-8ea0-d3d32f1cba83 None admin] PATH_INFO: `/v3/projects/47484d92809b4b9a846447c512be0a94` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:52.243914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f10f5-9039-4161-8ea0-d3d32f1cba83 None admin] RBAC: Authorizing `identity:get_project(project_id=47484d92809b4b9a846447c512be0a94)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:52.245379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f10f5-9039-4161-8ea0-d3d32f1cba83 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:52.247279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 228/450] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:52 2022] GET /identity/v3/projects/47484d92809b4b9a846447c512be0a94 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:52.248534 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0afac-0265-4e2f-a1a0-a40cbda6ee6d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:52.250455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:52.252991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f10f5-9039-4161-8ea0-d3d32f1cba83 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:52.299063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16d34a-c9ca-4d4d-b174-51b3c2f4d42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:52.300169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16d34a-c9ca-4d4d-b174-51b3c2f4d42f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:52.300432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16d34a-c9ca-4d4d-b174-51b3c2f4d42f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:52.300703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16d34a-c9ca-4d4d-b174-51b3c2f4d42f None admin] PATH_INFO: `/v3/projects/805a9d6e523c465393a8f47773ab2b23` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:52.301878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16d34a-c9ca-4d4d-b174-51b3c2f4d42f None admin] RBAC: Authorizing `identity:get_project(project_id=805a9d6e523c465393a8f47773ab2b23)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:52.303325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16d34a-c9ca-4d4d-b174-51b3c2f4d42f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:52.305253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 223/451] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:38:52 2022] GET /identity/v3/projects/805a9d6e523c465393a8f47773ab2b23 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:52.310544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16d34a-c9ca-4d4d-b174-51b3c2f4d42f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:52.367770 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f10f5-9039-4161-8ea0-d3d32f1cba83 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:52.407038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72a583-d4a3-4eee-8289-e3b77abac386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:52.407679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a583-d4a3-4eee-8289-e3b77abac386 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:52.407958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a583-d4a3-4eee-8289-e3b77abac386 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:52.408228 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a583-d4a3-4eee-8289-e3b77abac386 None admin] PATH_INFO: `/v3/groups/8a73142305e440fcaa40de1190133c24` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:52.409357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72a583-d4a3-4eee-8289-e3b77abac386 None admin] RBAC: Authorizing `identity:get_group(group_id=8a73142305e440fcaa40de1190133c24)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:52.410914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72a583-d4a3-4eee-8289-e3b77abac386 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:52.412509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 229/452] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:52 2022] GET /identity/v3/groups/8a73142305e440fcaa40de1190133c24 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:52.419429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72a583-d4a3-4eee-8289-e3b77abac386 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:52.434037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16d34a-c9ca-4d4d-b174-51b3c2f4d42f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:52.468895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:52.476571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249a9e2-3eb0-411d-90f5-dd99eeb795e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:52.477200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249a9e2-3eb0-411d-90f5-dd99eeb795e5 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:52.478054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249a9e2-3eb0-411d-90f5-dd99eeb795e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:52.478409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249a9e2-3eb0-411d-90f5-dd99eeb795e5 None admin] PATH_INFO: `/v3/groups/36e9ba2eae5542fd84e9624ca2971649` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:52.479439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249a9e2-3eb0-411d-90f5-dd99eeb795e5 None admin] RBAC: Authorizing `identity:get_group(group_id=36e9ba2eae5542fd84e9624ca2971649)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:52.480896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249a9e2-3eb0-411d-90f5-dd99eeb795e5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:52.482808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 224/453] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 3 14:38:52 2022] GET /identity/v3/groups/36e9ba2eae5542fd84e9624ca2971649 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:52.488204 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249a9e2-3eb0-411d-90f5-dd99eeb795e5 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:52.528134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:52.531204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72a583-d4a3-4eee-8289-e3b77abac386 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:52.563575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913f5ff-0274-47ea-a6e1-755152f50223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:52.564164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913f5ff-0274-47ea-a6e1-755152f50223 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:52.564436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913f5ff-0274-47ea-a6e1-755152f50223 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:52.564703 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913f5ff-0274-47ea-a6e1-755152f50223 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:52.565418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913f5ff-0274-47ea-a6e1-755152f50223 None admin] RBAC: Adding query filter params (group.id=8a73142305e440fcaa40de1190133c24, role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=47484d92809b4b9a846447c512be0a94) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:52.565744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913f5ff-0274-47ea-a6e1-755152f50223 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:52.567160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913f5ff-0274-47ea-a6e1-755152f50223 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:52.572658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 230/454] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 3 14:38:52 2022] GET /identity/v3/role_assignments?group.id=8a73142305e440fcaa40de1190133c24&scope.project.id=47484d92809b4b9a846447c512be0a94&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:52.585889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249a9e2-3eb0-411d-90f5-dd99eeb795e5 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:52.622472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15102b66-3a06-47ca-8156-d920d0dfdf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:52.623088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15102b66-3a06-47ca-8156-d920d0dfdf16 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:52.623366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15102b66-3a06-47ca-8156-d920d0dfdf16 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:52.623632 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15102b66-3a06-47ca-8156-d920d0dfdf16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:52.624379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15102b66-3a06-47ca-8156-d920d0dfdf16 None admin] RBAC: Adding query filter params (group.id=36e9ba2eae5542fd84e9624ca2971649, role.id=8cd21d388193490bac3aa386bcc18763, scope.project.id=805a9d6e523c465393a8f47773ab2b23) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:52.624685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15102b66-3a06-47ca-8156-d920d0dfdf16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:52.626193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15102b66-3a06-47ca-8156-d920d0dfdf16 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:52.632758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 225/455] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 3 14:38:52 2022] GET /identity/v3/role_assignments?group.id=36e9ba2eae5542fd84e9624ca2971649&scope.project.id=805a9d6e523c465393a8f47773ab2b23&role.id=8cd21d388193490bac3aa386bcc18763 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:55.244346 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5372367-5d98-41d9-bd4f-e82b0f4d3eaa None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:55.244871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5372367-5d98-41d9-bd4f-e82b0f4d3eaa None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:55.245212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5372367-5d98-41d9-bd4f-e82b0f4d3eaa None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:55.247006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 231/456] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:55.252600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1608e06-884e-4025-a7da-c926a1aed027 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:55.253392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1608e06-884e-4025-a7da-c926a1aed027 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:55.253878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1608e06-884e-4025-a7da-c926a1aed027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:55.283044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b1608e06-884e-4025-a7da-c926a1aed027 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:55.285278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b1608e06-884e-4025-a7da-c926a1aed027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:55.377366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 226/457] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:55.602940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fab87-7b7f-4c9b-9ce7-f739b55b99c6 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:55.603133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fab87-7b7f-4c9b-9ce7-f739b55b99c6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:55.603581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fab87-7b7f-4c9b-9ce7-f739b55b99c6 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:55.605285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 232/458] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:55.610372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8aec8-4b26-4dad-bb56-1d0100fdfe29 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:55.610806 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8aec8-4b26-4dad-bb56-1d0100fdfe29 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:55.611218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8aec8-4b26-4dad-bb56-1d0100fdfe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:55.619101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6921c7-aa94-4d60-b104-c95ecb5f2c75 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:55.619574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6921c7-aa94-4d60-b104-c95ecb5f2c75 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:55.620048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6921c7-aa94-4d60-b104-c95ecb5f2c75 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:55.621772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 233/459] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:38:55.634713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-52d8aec8-4b26-4dad-bb56-1d0100fdfe29 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:55.635919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-52d8aec8-4b26-4dad-bb56-1d0100fdfe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:55.639488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4876700-ebf9-4001-97b7-bd4a073a6abb None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:55.640135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4876700-ebf9-4001-97b7-bd4a073a6abb None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:55.640782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4876700-ebf9-4001-97b7-bd4a073a6abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:55.668503 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c4876700-ebf9-4001-97b7-bd4a073a6abb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:55.671345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c4876700-ebf9-4001-97b7-bd4a073a6abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:55.687798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:55.744411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 227/460] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:55.745618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8aec8-4b26-4dad-bb56-1d0100fdfe29 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:55.746368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 234/461] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:55.856079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8aec8-4b26-4dad-bb56-1d0100fdfe29 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:55.889393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaa823-7f30-4dc4-aa7c-038033847247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:55.890259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaa823-7f30-4dc4-aa7c-038033847247 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:55.890699 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaa823-7f30-4dc4-aa7c-038033847247 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:55.891109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaa823-7f30-4dc4-aa7c-038033847247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:55.901713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaa823-7f30-4dc4-aa7c-038033847247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:55.903326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaa823-7f30-4dc4-aa7c-038033847247 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-8ceaa823-7f30-4dc4-aa7c-038033847247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:38:55.918506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:55.919583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:38:55.920604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:38:55.920604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:38:55.920604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:55.921961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 228/462] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:55.928162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4876700-ebf9-4001-97b7-bd4a073a6abb None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:56.000064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:56.084144 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4876700-ebf9-4001-97b7-bd4a073a6abb None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:56.124957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaa823-7f30-4dc4-aa7c-038033847247 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:56.131820 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f01faa-b40f-48a4-9029-1987b509b3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:56.132431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f01faa-b40f-48a4-9029-1987b509b3af None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:56.132710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f01faa-b40f-48a4-9029-1987b509b3af None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:56.132981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f01faa-b40f-48a4-9029-1987b509b3af None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:56.133698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f01faa-b40f-48a4-9029-1987b509b3af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:56.134002 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f01faa-b40f-48a4-9029-1987b509b3af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:56.135382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f01faa-b40f-48a4-9029-1987b509b3af None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:56.144383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:56.154976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 235/463] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:56.160329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f01faa-b40f-48a4-9029-1987b509b3af None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:56.263262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaa823-7f30-4dc4-aa7c-038033847247 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:56.282079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:56.285718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f01faa-b40f-48a4-9029-1987b509b3af None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:56.312350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beed191-11b1-49ab-b85b-c6e0645b8700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:56.313334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beed191-11b1-49ab-b85b-c6e0645b8700 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:56.313833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beed191-11b1-49ab-b85b-c6e0645b8700 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:56.314324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beed191-11b1-49ab-b85b-c6e0645b8700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:56.323280 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de577aba-508d-40bc-b0cf-f5a1a7e7cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:56.323882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de577aba-508d-40bc-b0cf-f5a1a7e7cdc6 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:56.324157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de577aba-508d-40bc-b0cf-f5a1a7e7cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:56.324426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de577aba-508d-40bc-b0cf-f5a1a7e7cdc6 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:56.325065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de577aba-508d-40bc-b0cf-f5a1a7e7cdc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:56.326563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de577aba-508d-40bc-b0cf-f5a1a7e7cdc6 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:56.327986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beed191-11b1-49ab-b85b-c6e0645b8700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:56.329684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beed191-11b1-49ab-b85b-c6e0645b8700 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-0beed191-11b1-49ab-b85b-c6e0645b8700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:38:56.349030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:56.349997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:38:56.350831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:38:56.350831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:38:56.350831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:56.351389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 236/464] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:38:56 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:38:56.352161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de577aba-508d-40bc-b0cf-f5a1a7e7cdc6 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:56.358018 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 229/465] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:56.360949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beed191-11b1-49ab-b85b-c6e0645b8700 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:56.452144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:56.478784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de577aba-508d-40bc-b0cf-f5a1a7e7cdc6 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:56.497591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beed191-11b1-49ab-b85b-c6e0645b8700 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:56.517163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15333ca-5dea-41dc-a6c5-cb16915b1ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:56.517972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15333ca-5dea-41dc-a6c5-cb16915b1ae7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:56.518253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15333ca-5dea-41dc-a6c5-cb16915b1ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:56.518544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15333ca-5dea-41dc-a6c5-cb16915b1ae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:56.531518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15333ca-5dea-41dc-a6c5-cb16915b1ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:56.533164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15333ca-5dea-41dc-a6c5-cb16915b1ae7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:56.538417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835a4de-435e-43fb-a71e-28122f55ba8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:56.539393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835a4de-435e-43fb-a71e-28122f55ba8e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:56.540015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835a4de-435e-43fb-a71e-28122f55ba8e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:56.541149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835a4de-435e-43fb-a71e-28122f55ba8e None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:56.542078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835a4de-435e-43fb-a71e-28122f55ba8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:56.543156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835a4de-435e-43fb-a71e-28122f55ba8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:56.545286 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835a4de-435e-43fb-a71e-28122f55ba8e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-c15333ca-5dea-41dc-a6c5-cb16915b1ae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:38:56.550887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:56.551771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:38:56.552591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:38:56.552591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:38:56.552591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:38:56.552591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 237/466] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:56.559286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15333ca-5dea-41dc-a6c5-cb16915b1ae7 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:56.561591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 230/467] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:56.567291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835a4de-435e-43fb-a71e-28122f55ba8e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:56.572586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:56.684355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15333ca-5dea-41dc-a6c5-cb16915b1ae7 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:56.686552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835a4de-435e-43fb-a71e-28122f55ba8e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:56.716897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:56.726135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d91e5-fc58-4dfd-8c2b-c687e2a1ceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:56.726749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d91e5-fc58-4dfd-8c2b-c687e2a1ceb2 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:56.727030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d91e5-fc58-4dfd-8c2b-c687e2a1ceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:56.727332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d91e5-fc58-4dfd-8c2b-c687e2a1ceb2 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:56.727981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d91e5-fc58-4dfd-8c2b-c687e2a1ceb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:56.729414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d91e5-fc58-4dfd-8c2b-c687e2a1ceb2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:56.732720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78b80b-58a4-40b5-bd0b-cbee2aab1554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:56.733586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78b80b-58a4-40b5-bd0b-cbee2aab1554 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:56.734065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78b80b-58a4-40b5-bd0b-cbee2aab1554 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:56.734531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78b80b-58a4-40b5-bd0b-cbee2aab1554 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:56.735374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78b80b-58a4-40b5-bd0b-cbee2aab1554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:56.735903 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78b80b-58a4-40b5-bd0b-cbee2aab1554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:56.737488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78b80b-58a4-40b5-bd0b-cbee2aab1554 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:56.752315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 231/468] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:38:56 2022] POST /identity/v3/users => generated 248 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:38:56.753724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f51bb-727e-42ad-8c73-51e51af907cc None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:56.753974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 238/469] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:38:56.754100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f51bb-727e-42ad-8c73-51e51af907cc None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:56.754321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f51bb-727e-42ad-8c73-51e51af907cc None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:56.760220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78b80b-58a4-40b5-bd0b-cbee2aab1554 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:56.760500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 232/470] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:56.762238 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b727a1-e188-4f6b-99a7-63724a5b6320 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:56.762521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b727a1-e188-4f6b-99a7-63724a5b6320 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:56.762793 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b727a1-e188-4f6b-99a7-63724a5b6320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:56.785445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a7b727a1-e188-4f6b-99a7-63724a5b6320 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:56.786850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a7b727a1-e188-4f6b-99a7-63724a5b6320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:56.842799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:56.875425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 233/471] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:56.897745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78b80b-58a4-40b5-bd0b-cbee2aab1554 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:56.960364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11488945-d931-4069-a67e-696e2aababc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:56.961004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11488945-d931-4069-a67e-696e2aababc6 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:56.961282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11488945-d931-4069-a67e-696e2aababc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:56.961651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11488945-d931-4069-a67e-696e2aababc6 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:56.962335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11488945-d931-4069-a67e-696e2aababc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:56.963837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11488945-d931-4069-a67e-696e2aababc6 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:56.989159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 239/472] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:38:56 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:38:57.076850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6e8b4-0e39-4ebf-acf8-98aa58997b81 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:57.077389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6e8b4-0e39-4ebf-acf8-98aa58997b81 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:57.077934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6e8b4-0e39-4ebf-acf8-98aa58997b81 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:57.079544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 234/473] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:57.085745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6690c-e4b0-432e-a207-3731846f9b4e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:57.086033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6690c-e4b0-432e-a207-3731846f9b4e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:57.086302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6690c-e4b0-432e-a207-3731846f9b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:57.108392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-93c6690c-e4b0-432e-a207-3731846f9b4e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:57.109652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-93c6690c-e4b0-432e-a207-3731846f9b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:57.186417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 240/474] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:57.227280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6e8b4-0e39-4ebf-acf8-98aa58997b81 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:57.355159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6e8b4-0e39-4ebf-acf8-98aa58997b81 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:57.370371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:57.395214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff582e-4795-41ce-9528-b68525b6d73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:57.395856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff582e-4795-41ce-9528-b68525b6d73d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:57.396135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff582e-4795-41ce-9528-b68525b6d73d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:57.396405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff582e-4795-41ce-9528-b68525b6d73d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:57.408663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff582e-4795-41ce-9528-b68525b6d73d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:57.410176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff582e-4795-41ce-9528-b68525b6d73d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-beff582e-4795-41ce-9528-b68525b6d73d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:38:57.426096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:57.427002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:38:57.428525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:38:57.428525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:38:57.428525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:57.428525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 235/475] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:57.433902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6690c-e4b0-432e-a207-3731846f9b4e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:57.522953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:57.558799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff582e-4795-41ce-9528-b68525b6d73d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:57.588048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6690c-e4b0-432e-a207-3731846f9b4e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:57.663213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbe3b4-a09d-4819-a99b-fd23c26f81df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:57.663825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbe3b4-a09d-4819-a99b-fd23c26f81df None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:57.664105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbe3b4-a09d-4819-a99b-fd23c26f81df None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:57.664439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbe3b4-a09d-4819-a99b-fd23c26f81df None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:57.665662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbe3b4-a09d-4819-a99b-fd23c26f81df None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:57.666637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbe3b4-a09d-4819-a99b-fd23c26f81df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:57.668309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbe3b4-a09d-4819-a99b-fd23c26f81df None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:57.695731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 241/476] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:57.703234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbe3b4-a09d-4819-a99b-fd23c26f81df None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:57.722311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff582e-4795-41ce-9528-b68525b6d73d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:57.793070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512470b3-f60b-4a86-bfc7-b471e43bb1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:57.794004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512470b3-f60b-4a86-bfc7-b471e43bb1c5 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:57.794423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512470b3-f60b-4a86-bfc7-b471e43bb1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:57.794898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512470b3-f60b-4a86-bfc7-b471e43bb1c5 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:57.797061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512470b3-f60b-4a86-bfc7-b471e43bb1c5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:57.800931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512470b3-f60b-4a86-bfc7-b471e43bb1c5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:57.816461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.sql.core [None req-512470b3-f60b-4a86-bfc7-b471e43bb1c5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 14:38:57.816461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 03 14:38:57.816461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [parameters: {'id': 'ae3235a4b6dd4aa5844bbf3102d61427', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 03 14:38:57.816461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87830) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-512470b3-f60b-4a86-bfc7-b471e43bb1c5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application conn.query(q) Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application result.read() Oct 03 14:38:57.838239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return ref.to_dict() Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 14:38:57.839131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application next(self.gen) Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application yield resource Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application next(self.gen) Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise self.value Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application session.commit() Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 14:38:57.840314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self.session.flush() Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application compat.raise_( Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 14:38:57.841145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 14:38:57.842010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application conn.query(q) Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application result.read() Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 14:38:57.842780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [parameters: {'id': 'ae3235a4b6dd4aa5844bbf3102d61427', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:57.843564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 03 14:38:57.844315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:57.845119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 236/477] 10.222.0.46 () {60 vars in 1131 bytes} [Mon Oct 3 14:38:57 2022] POST /identity/v3/roles => generated 144 bytes in 281 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:57.852048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512470b3-f60b-4a86-bfc7-b471e43bb1c5 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:57.920015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbe3b4-a09d-4819-a99b-fd23c26f81df None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:57.925848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:57.944587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:57.990188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75539811-8cf9-46b0-98ce-8a480546a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:57.990819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75539811-8cf9-46b0-98ce-8a480546a77b None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:57.991097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75539811-8cf9-46b0-98ce-8a480546a77b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:57.991391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75539811-8cf9-46b0-98ce-8a480546a77b None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:57.992050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75539811-8cf9-46b0-98ce-8a480546a77b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:57.994481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75539811-8cf9-46b0-98ce-8a480546a77b None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:58.034169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 242/478] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:38:57 2022] POST /identity/v3/users => generated 249 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:38:58.040272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512470b3-f60b-4a86-bfc7-b471e43bb1c5 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:58.078652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05b2b9-ab68-4ed0-aa1e-2a9aaeb00208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:58.079400 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05b2b9-ab68-4ed0-aa1e-2a9aaeb00208 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:58.079748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05b2b9-ab68-4ed0-aa1e-2a9aaeb00208 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:58.080087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05b2b9-ab68-4ed0-aa1e-2a9aaeb00208 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:58.092773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05b2b9-ab68-4ed0-aa1e-2a9aaeb00208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-1c05b2b9-ab68-4ed0-aa1e-2a9aaeb00208 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:58.098598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:58.099529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:58.100574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 237/479] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:38:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:38:58.110608 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75539811-8cf9-46b0-98ce-8a480546a77b None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:58.268417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75539811-8cf9-46b0-98ce-8a480546a77b None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:58.319309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c6d22-4415-4c11-b2d7-e79db9cee0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:58.319920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c6d22-4415-4c11-b2d7-e79db9cee0dc None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:58.320193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c6d22-4415-4c11-b2d7-e79db9cee0dc None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:58.320455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c6d22-4415-4c11-b2d7-e79db9cee0dc None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:58.321676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c6d22-4415-4c11-b2d7-e79db9cee0dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:58.321976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c6d22-4415-4c11-b2d7-e79db9cee0dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:58.323345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c6d22-4415-4c11-b2d7-e79db9cee0dc None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:58.338147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 243/480] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:38:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:59.040981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06916e0-1f81-4d9f-af41-24f4659fd49f None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:59.041539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06916e0-1f81-4d9f-af41-24f4659fd49f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:59.041539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06916e0-1f81-4d9f-af41-24f4659fd49f None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:59.043136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 238/481] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:38:59.052163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dcced-0461-4c91-8d5f-882277b7dada None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:59.052572 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dcced-0461-4c91-8d5f-882277b7dada None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:59.052760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dcced-0461-4c91-8d5f-882277b7dada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:59.077686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-fb6dcced-0461-4c91-8d5f-882277b7dada None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:59.078867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-fb6dcced-0461-4c91-8d5f-882277b7dada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:59.113227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:59.167727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 244/482] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:38:59.479391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06916e0-1f81-4d9f-af41-24f4659fd49f None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:59.550294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f03e7-501b-476f-909c-dac4e88e93ee None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:59.550889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f03e7-501b-476f-909c-dac4e88e93ee None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:59.551357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f03e7-501b-476f-909c-dac4e88e93ee None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:59.553149 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 245/483] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:38:59.558756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e86ff-f2d0-4bf4-ba92-e3fcdcb92d19 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:59.559257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e86ff-f2d0-4bf4-ba92-e3fcdcb92d19 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:59.559740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e86ff-f2d0-4bf4-ba92-e3fcdcb92d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:59.579684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-4c8e86ff-f2d0-4bf4-ba92-e3fcdcb92d19 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:59.581838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4c8e86ff-f2d0-4bf4-ba92-e3fcdcb92d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:59.610625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:59.632946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06916e0-1f81-4d9f-af41-24f4659fd49f None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:59.668375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e32b20-9c9b-48c9-ba48-421eaf646f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:59.669320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e32b20-9c9b-48c9-ba48-421eaf646f29 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:59.669768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e32b20-9c9b-48c9-ba48-421eaf646f29 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:59.670178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e32b20-9c9b-48c9-ba48-421eaf646f29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:59.681255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e32b20-9c9b-48c9-ba48-421eaf646f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-80e32b20-9c9b-48c9-ba48-421eaf646f29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:38:59.694688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:38:59.696299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:38:59.697287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:38:59.697957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 246/484] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:38:59.699443 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 239/485] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:38:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:38:59.705131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e86ff-f2d0-4bf4-ba92-e3fcdcb92d19 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:59.724851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f622a-ba88-4e1f-a740-0ad31f8cbb24 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:59.725447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f622a-ba88-4e1f-a740-0ad31f8cbb24 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:59.725940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f622a-ba88-4e1f-a740-0ad31f8cbb24 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:59.727987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 240/486] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:38:59.733573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924339f9-8dea-4663-abec-fb53c8742e6a None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:59.734113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924339f9-8dea-4663-abec-fb53c8742e6a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:59.734581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924339f9-8dea-4663-abec-fb53c8742e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:59.768848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-924339f9-8dea-4663-abec-fb53c8742e6a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:38:59.770091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-924339f9-8dea-4663-abec-fb53c8742e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:38:59.801951 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:38:59.813528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e86ff-f2d0-4bf4-ba92-e3fcdcb92d19 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:38:59.870118 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c1d48-5d0b-44ae-95e0-2d6d9ab269bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:38:59.870746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c1d48-5d0b-44ae-95e0-2d6d9ab269bb None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:38:59.871026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c1d48-5d0b-44ae-95e0-2d6d9ab269bb None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:38:59.871295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c1d48-5d0b-44ae-95e0-2d6d9ab269bb None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:38:59.871986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c1d48-5d0b-44ae-95e0-2d6d9ab269bb None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:38:59.872405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c1d48-5d0b-44ae-95e0-2d6d9ab269bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:38:59.873829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c1d48-5d0b-44ae-95e0-2d6d9ab269bb None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:38:59.892275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 247/487] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:38:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:38:59.898277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c1d48-5d0b-44ae-95e0-2d6d9ab269bb None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:38:59.917007 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 241/488] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:39:00.012543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924339f9-8dea-4663-abec-fb53c8742e6a None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:00.064373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c1d48-5d0b-44ae-95e0-2d6d9ab269bb None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:00.151322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c282e1b-e975-4b4e-adcc-6038155d56de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:00.151954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c282e1b-e975-4b4e-adcc-6038155d56de None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:00.152229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c282e1b-e975-4b4e-adcc-6038155d56de None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:00.152504 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c282e1b-e975-4b4e-adcc-6038155d56de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:00.165836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:00.172292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c282e1b-e975-4b4e-adcc-6038155d56de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:00.175794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c282e1b-e975-4b4e-adcc-6038155d56de None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-7c282e1b-e975-4b4e-adcc-6038155d56de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:00.202479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:00.203599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:00.204640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:00.204640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:00.204640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:00.204640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 248/489] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:00.205070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c282e1b-e975-4b4e-adcc-6038155d56de None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:00.206070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924339f9-8dea-4663-abec-fb53c8742e6a None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:00.215271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:00.250474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fa32d-1166-4406-8470-902ea368ec6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:00.251110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fa32d-1166-4406-8470-902ea368ec6b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:00.251390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fa32d-1166-4406-8470-902ea368ec6b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:00.251664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fa32d-1166-4406-8470-902ea368ec6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:00.262684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fa32d-1166-4406-8470-902ea368ec6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-9d5fa32d-1166-4406-8470-902ea368ec6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:00.267543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:00.268428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:00.269229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 242/490] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:00.269991 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fa32d-1166-4406-8470-902ea368ec6b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:00.367468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c282e1b-e975-4b4e-adcc-6038155d56de None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:00.388790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fa32d-1166-4406-8470-902ea368ec6b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:00.414782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10908c4e-7a68-4feb-b550-29d104a4f077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:00.415922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10908c4e-7a68-4feb-b550-29d104a4f077 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:00.416205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10908c4e-7a68-4feb-b550-29d104a4f077 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:00.416782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10908c4e-7a68-4feb-b550-29d104a4f077 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:00.426775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dec79f-5503-4eb0-bf8a-7c58572d8173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:00.427451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dec79f-5503-4eb0-bf8a-7c58572d8173 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:00.427801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dec79f-5503-4eb0-bf8a-7c58572d8173 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:00.428142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dec79f-5503-4eb0-bf8a-7c58572d8173 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:00.428858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dec79f-5503-4eb0-bf8a-7c58572d8173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:00.429200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10908c4e-7a68-4feb-b550-29d104a4f077 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:00.429314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dec79f-5503-4eb0-bf8a-7c58572d8173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:00.430769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dec79f-5503-4eb0-bf8a-7c58572d8173 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-10908c4e-7a68-4feb-b550-29d104a4f077 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:00.436211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:00.437156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:00.437982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:00.439533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:00.439638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 249/491] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:00.439638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10908c4e-7a68-4feb-b550-29d104a4f077 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:00.445412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 243/492] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:00.446811 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dec79f-5503-4eb0-bf8a-7c58572d8173 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:00.528004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:00.562453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dec79f-5503-4eb0-bf8a-7c58572d8173 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:00.571246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10908c4e-7a68-4feb-b550-29d104a4f077 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:00.606556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60857a-8b31-410a-9baf-285bd2274b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:00.607021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60857a-8b31-410a-9baf-285bd2274b1e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:00.607302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60857a-8b31-410a-9baf-285bd2274b1e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:00.607588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60857a-8b31-410a-9baf-285bd2274b1e None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:00.608252 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60857a-8b31-410a-9baf-285bd2274b1e None admin] RBAC: Adding query filter params (name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:00.608557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60857a-8b31-410a-9baf-285bd2274b1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:00.609972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60857a-8b31-410a-9baf-285bd2274b1e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:00.617017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f51c00-0319-4a2e-8a24-dec73ab472a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:00.618153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51c00-0319-4a2e-8a24-dec73ab472a9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:00.618593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51c00-0319-4a2e-8a24-dec73ab472a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:00.619006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51c00-0319-4a2e-8a24-dec73ab472a9 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:00.619478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 244/493] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:00.620186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60857a-8b31-410a-9baf-285bd2274b1e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:00.620407 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f51c00-0319-4a2e-8a24-dec73ab472a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:00.620863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f51c00-0319-4a2e-8a24-dec73ab472a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:00.622398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f51c00-0319-4a2e-8a24-dec73ab472a9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:00.634991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 250/494] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:00.636302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f51c00-0319-4a2e-8a24-dec73ab472a9 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:00.680655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:00.727826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60857a-8b31-410a-9baf-285bd2274b1e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:00.765910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a478e9-9196-4bea-99b7-bc16496b6175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:00.766563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a478e9-9196-4bea-99b7-bc16496b6175 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:00.766889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a478e9-9196-4bea-99b7-bc16496b6175 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:00.767264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a478e9-9196-4bea-99b7-bc16496b6175 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:00.775838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f51c00-0319-4a2e-8a24-dec73ab472a9 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:00.776029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a478e9-9196-4bea-99b7-bc16496b6175 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:00.777399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a478e9-9196-4bea-99b7-bc16496b6175 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-a6a478e9-9196-4bea-99b7-bc16496b6175 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:00.791957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:00.792923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:00.794277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 245/495] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 3 14:39:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:00.794841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a478e9-9196-4bea-99b7-bc16496b6175 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:00.812302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:00.823219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34972b-313d-41bf-b6d5-b24b0deab8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:00.824130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34972b-313d-41bf-b6d5-b24b0deab8c4 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:00.824612 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34972b-313d-41bf-b6d5-b24b0deab8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:00.825086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34972b-313d-41bf-b6d5-b24b0deab8c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:00.837002 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34972b-313d-41bf-b6d5-b24b0deab8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:00.838797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34972b-313d-41bf-b6d5-b24b0deab8c4 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-4a34972b-313d-41bf-b6d5-b24b0deab8c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:00.853993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:00.854930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:00.855791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:00.855791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:00.855791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:00.857855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 251/496] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:00.858936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34972b-313d-41bf-b6d5-b24b0deab8c4 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:00.903079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a478e9-9196-4bea-99b7-bc16496b6175 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:00.925256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:00.942522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3b367-94eb-4a7b-b1a8-b66c0ff28934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:00.943526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3b367-94eb-4a7b-b1a8-b66c0ff28934 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:00.943818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3b367-94eb-4a7b-b1a8-b66c0ff28934 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:00.944096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3b367-94eb-4a7b-b1a8-b66c0ff28934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:00.955208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3b367-94eb-4a7b-b1a8-b66c0ff28934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:00.956669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3b367-94eb-4a7b-b1a8-b66c0ff28934 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:00.962528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34972b-313d-41bf-b6d5-b24b0deab8c4 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-d5c3b367-94eb-4a7b-b1a8-b66c0ff28934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:00.971625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:00.972549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:00.978623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:00.978623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:00.978623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:00.978623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 246/497] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:00.978623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3b367-94eb-4a7b-b1a8-b66c0ff28934 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:01.000522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71d0f2-7dc9-4acf-ace5-c013282a7a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:01.001127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71d0f2-7dc9-4acf-ace5-c013282a7a3c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.001402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71d0f2-7dc9-4acf-ace5-c013282a7a3c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.001700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71d0f2-7dc9-4acf-ace5-c013282a7a3c None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.002424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71d0f2-7dc9-4acf-ace5-c013282a7a3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:01.002839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71d0f2-7dc9-4acf-ace5-c013282a7a3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:01.004386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71d0f2-7dc9-4acf-ace5-c013282a7a3c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:01.043130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 252/498] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 3 14:39:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:01.044493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c0816-6dc4-4fc5-ab3f-e55d61bb3871 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.044772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c0816-6dc4-4fc5-ab3f-e55d61bb3871 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.045043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c0816-6dc4-4fc5-ab3f-e55d61bb3871 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.046405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 253/499] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:01.046938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c0816-6dc4-4fc5-ab3f-e55d61bb3871 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:01.086457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3b367-94eb-4a7b-b1a8-b66c0ff28934 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:01.087287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:01.124537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb775ab6-fe58-42c5-ae75-a6cbcd063581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:01.125562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb775ab6-fe58-42c5-ae75-a6cbcd063581 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.125882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb775ab6-fe58-42c5-ae75-a6cbcd063581 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.126167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb775ab6-fe58-42c5-ae75-a6cbcd063581 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.126849 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb775ab6-fe58-42c5-ae75-a6cbcd063581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:01.127160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb775ab6-fe58-42c5-ae75-a6cbcd063581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:01.128573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb775ab6-fe58-42c5-ae75-a6cbcd063581 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:01.143744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 247/500] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:01.144867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb775ab6-fe58-42c5-ae75-a6cbcd063581 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:01.168113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c0816-6dc4-4fc5-ab3f-e55d61bb3871 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:01.178001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:01.206704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360f810-6a8e-41de-b60a-f568dec78955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:01.207654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360f810-6a8e-41de-b60a-f568dec78955 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.208152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360f810-6a8e-41de-b60a-f568dec78955 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.208489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360f810-6a8e-41de-b60a-f568dec78955 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.209183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360f810-6a8e-41de-b60a-f568dec78955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:01.209433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360f810-6a8e-41de-b60a-f568dec78955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:01.210837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360f810-6a8e-41de-b60a-f568dec78955 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:01.226153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 254/501] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:01.226153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed33bb-1e77-49c8-b75d-28a74eb8243e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.226153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed33bb-1e77-49c8-b75d-28a74eb8243e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.226153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed33bb-1e77-49c8-b75d-28a74eb8243e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.242103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ffed33bb-1e77-49c8-b75d-28a74eb8243e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:01.243891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ffed33bb-1e77-49c8-b75d-28a74eb8243e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:01.253145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb775ab6-fe58-42c5-ae75-a6cbcd063581 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:01.285382 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2682f-7d03-4da0-9799-9ab296fb6fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:01.286010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2682f-7d03-4da0-9799-9ab296fb6fa7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.286287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2682f-7d03-4da0-9799-9ab296fb6fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.286555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2682f-7d03-4da0-9799-9ab296fb6fa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.308837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 255/502] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:01.309707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed33bb-1e77-49c8-b75d-28a74eb8243e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:01.335496 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:01.358141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2682f-7d03-4da0-9799-9ab296fb6fa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:01.359665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2682f-7d03-4da0-9799-9ab296fb6fa7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:01.391209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:01.414056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed33bb-1e77-49c8-b75d-28a74eb8243e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:01.449953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7c30e-9045-44da-b5a3-6c594cca173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:01.450498 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7c30e-9045-44da-b5a3-6c594cca173a None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.450770 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7c30e-9045-44da-b5a3-6c594cca173a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.451446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7c30e-9045-44da-b5a3-6c594cca173a None admin] PATH_INFO: `/v3/users/glance` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.458462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7c30e-9045-44da-b5a3-6c594cca173a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:01.459899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7c30e-9045-44da-b5a3-6c594cca173a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:01.469070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 248/503] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:39:01.469070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2682f-7d03-4da0-9799-9ab296fb6fa7 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-81d7c30e-9045-44da-b5a3-6c594cca173a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:01.473465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:01.474409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:01.475300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 256/504] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:01.475854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6996e-ba9b-4e99-9f01-0f41010142ad None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.476974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6996e-ba9b-4e99-9f01-0f41010142ad None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.476974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6996e-ba9b-4e99-9f01-0f41010142ad None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.481844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 257/505] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:01.482583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6996e-ba9b-4e99-9f01-0f41010142ad None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:01.569856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:01.577729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2682f-7d03-4da0-9799-9ab296fb6fa7 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:01.587085 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6996e-ba9b-4e99-9f01-0f41010142ad None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:01.609221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:01.615348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1c20d-ee02-4b20-97ce-77cae2ed750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:01.615973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1c20d-ee02-4b20-97ce-77cae2ed750e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.616246 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1c20d-ee02-4b20-97ce-77cae2ed750e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.616511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1c20d-ee02-4b20-97ce-77cae2ed750e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.621010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e856aec-55df-4734-b2eb-9711a3bb0f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:01.621662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856aec-55df-4734-b2eb-9711a3bb0f2c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.621940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856aec-55df-4734-b2eb-9711a3bb0f2c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.622207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856aec-55df-4734-b2eb-9711a3bb0f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.623907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1c20d-ee02-4b20-97ce-77cae2ed750e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:01.625338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1c20d-ee02-4b20-97ce-77cae2ed750e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:01.632126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e856aec-55df-4734-b2eb-9711a3bb0f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:01.634247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e856aec-55df-4734-b2eb-9711a3bb0f2c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-fcc1c20d-ee02-4b20-97ce-77cae2ed750e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:01.637780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:01.638844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:01.639714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 249/506] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:01.640260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1c20d-ee02-4b20-97ce-77cae2ed750e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-1e856aec-55df-4734-b2eb-9711a3bb0f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:01.648757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:01.649643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:01.650481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:01.650481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:01.650481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:01.650481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 258/507] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:01.651025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7f8fe-57d2-4c13-be3a-a138aba5081b None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.651305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7f8fe-57d2-4c13-be3a-a138aba5081b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.651576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7f8fe-57d2-4c13-be3a-a138aba5081b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.668658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f3a7f8fe-57d2-4c13-be3a-a138aba5081b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:01.669876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f3a7f8fe-57d2-4c13-be3a-a138aba5081b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:01.734945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 259/508] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:01.735822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7f8fe-57d2-4c13-be3a-a138aba5081b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:01.741686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1c20d-ee02-4b20-97ce-77cae2ed750e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:01.778833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8312bb0-df68-4556-ba58-d310c035d6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:01.779445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8312bb0-df68-4556-ba58-d310c035d6fc None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.779729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8312bb0-df68-4556-ba58-d310c035d6fc None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.780006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8312bb0-df68-4556-ba58-d310c035d6fc None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.780740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8312bb0-df68-4556-ba58-d310c035d6fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:01.781052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8312bb0-df68-4556-ba58-d310c035d6fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:01.782517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8312bb0-df68-4556-ba58-d310c035d6fc None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:01.808909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:01.823935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 250/509] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 3 14:39:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:01.823935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8312bb0-df68-4556-ba58-d310c035d6fc None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:01.848972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7f8fe-57d2-4c13-be3a-a138aba5081b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:01.874953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:01.888759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a00bd7-0f39-4e1c-ada5-af84bb19d2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:01.889383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a00bd7-0f39-4e1c-ada5-af84bb19d2ff None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.889688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a00bd7-0f39-4e1c-ada5-af84bb19d2ff None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.889981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a00bd7-0f39-4e1c-ada5-af84bb19d2ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.900320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a00bd7-0f39-4e1c-ada5-af84bb19d2ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:01.901796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a00bd7-0f39-4e1c-ada5-af84bb19d2ff None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-33a00bd7-0f39-4e1c-ada5-af84bb19d2ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:01.916804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:01.917725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:01.918619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:01.918619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:01.918619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:01.918619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 260/510] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:01.918619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a00bd7-0f39-4e1c-ada5-af84bb19d2ff None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:01.932397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8312bb0-df68-4556-ba58-d310c035d6fc None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:01.968863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad80f63-9c79-44c2-8c53-363697911444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:01.969595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad80f63-9c79-44c2-8c53-363697911444 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:01.969878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad80f63-9c79-44c2-8c53-363697911444 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:01.970154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad80f63-9c79-44c2-8c53-363697911444 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:01.970912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad80f63-9c79-44c2-8c53-363697911444 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:01.971217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad80f63-9c79-44c2-8c53-363697911444 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:01.972654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad80f63-9c79-44c2-8c53-363697911444 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:01.990519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 251/511] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 3 14:39:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:01.991018 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad80f63-9c79-44c2-8c53-363697911444 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:02.032814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a00bd7-0f39-4e1c-ada5-af84bb19d2ff None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.052958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:02.067971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a83a5f-ae18-46a5-ae73-49db96609594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:02.068573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a83a5f-ae18-46a5-ae73-49db96609594 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:02.068850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a83a5f-ae18-46a5-ae73-49db96609594 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:02.069137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a83a5f-ae18-46a5-ae73-49db96609594 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:02.069791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a83a5f-ae18-46a5-ae73-49db96609594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:02.070093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a83a5f-ae18-46a5-ae73-49db96609594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:02.071489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a83a5f-ae18-46a5-ae73-49db96609594 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:02.083041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 261/512] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:02.083817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a83a5f-ae18-46a5-ae73-49db96609594 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:02.096452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad80f63-9c79-44c2-8c53-363697911444 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.106453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:02.131517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be7e58-3e1d-4c81-b159-6407fad5defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:02.132124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7e58-3e1d-4c81-b159-6407fad5defa None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:02.132400 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7e58-3e1d-4c81-b159-6407fad5defa None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:02.132672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7e58-3e1d-4c81-b159-6407fad5defa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:02.181818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a83a5f-ae18-46a5-ae73-49db96609594 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.200816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be7e58-3e1d-4c81-b159-6407fad5defa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:02.202772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be7e58-3e1d-4c81-b159-6407fad5defa None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:02.214724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a198962-91a7-4ff2-a154-1a71dbcd94c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:02.215319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a198962-91a7-4ff2-a154-1a71dbcd94c2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:02.215594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a198962-91a7-4ff2-a154-1a71dbcd94c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:02.215860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a198962-91a7-4ff2-a154-1a71dbcd94c2 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:02.216490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a198962-91a7-4ff2-a154-1a71dbcd94c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:02.216786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a198962-91a7-4ff2-a154-1a71dbcd94c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:02.218176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a198962-91a7-4ff2-a154-1a71dbcd94c2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:02.228974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 262/513] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:02.229925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a198962-91a7-4ff2-a154-1a71dbcd94c2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:02.270856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:02.305561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 252/514] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:39:02.306338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be7e58-3e1d-4c81-b159-6407fad5defa None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:02.318052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:02.333925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a198962-91a7-4ff2-a154-1a71dbcd94c2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.367102 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db0548-be0d-4854-a174-0ebac4d785a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:02.368237 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0548-be0d-4854-a174-0ebac4d785a1 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:02.368516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0548-be0d-4854-a174-0ebac4d785a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:02.368784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0548-be0d-4854-a174-0ebac4d785a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:02.404998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be7e58-3e1d-4c81-b159-6407fad5defa None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.433937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0548-be0d-4854-a174-0ebac4d785a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:02.435404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0548-be0d-4854-a174-0ebac4d785a1 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:02.437958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9fbbb-b042-4855-83d9-53c89903255b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:02.438587 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9fbbb-b042-4855-83d9-53c89903255b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:02.438868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9fbbb-b042-4855-83d9-53c89903255b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:02.439141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9fbbb-b042-4855-83d9-53c89903255b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:02.448916 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9fbbb-b042-4855-83d9-53c89903255b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-7cf9fbbb-b042-4855-83d9-53c89903255b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:02.453677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:02.454703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 253/515] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:02.458882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9fbbb-b042-4855-83d9-53c89903255b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:02.479238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:02.521155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:02.537431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 263/516] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:39:02.538277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db0548-be0d-4854-a174-0ebac4d785a1 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:02.559042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9fbbb-b042-4855-83d9-53c89903255b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.592610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41503adf-dacf-4747-a74b-0f52fdb1a9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:02.593236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41503adf-dacf-4747-a74b-0f52fdb1a9f3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:02.593544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41503adf-dacf-4747-a74b-0f52fdb1a9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:02.593823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41503adf-dacf-4747-a74b-0f52fdb1a9f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:02.600177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41503adf-dacf-4747-a74b-0f52fdb1a9f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:02.601661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41503adf-dacf-4747-a74b-0f52fdb1a9f3 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-41503adf-dacf-4747-a74b-0f52fdb1a9f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:02.613622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:02.614501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:02.615631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:02.615631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:02.615631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:02.615631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:02.615631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:02.615631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:02.615631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:02.615631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:02.615631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 254/517] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:02.616383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41503adf-dacf-4747-a74b-0f52fdb1a9f3 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:02.636416 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db0548-be0d-4854-a174-0ebac4d785a1 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.669952 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558656d9-d8c6-49f4-85a9-55bace7423c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:02.670574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558656d9-d8c6-49f4-85a9-55bace7423c0 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:02.670849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558656d9-d8c6-49f4-85a9-55bace7423c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:02.671119 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558656d9-d8c6-49f4-85a9-55bace7423c0 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:02.671751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558656d9-d8c6-49f4-85a9-55bace7423c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:02.673179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558656d9-d8c6-49f4-85a9-55bace7423c0 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:02.683791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:02.694426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 264/518] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:39:02 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:39:02.695214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558656d9-d8c6-49f4-85a9-55bace7423c0 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:02.729878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41503adf-dacf-4747-a74b-0f52fdb1a9f3 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.761131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:02.770734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1dd386-980b-45d9-8616-af5ba1dd4da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:02.771550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dd386-980b-45d9-8616-af5ba1dd4da0 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:02.771783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dd386-980b-45d9-8616-af5ba1dd4da0 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:02.772057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dd386-980b-45d9-8616-af5ba1dd4da0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:02.784678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1dd386-980b-45d9-8616-af5ba1dd4da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:02.786370 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1dd386-980b-45d9-8616-af5ba1dd4da0 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-9c1dd386-980b-45d9-8616-af5ba1dd4da0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:02.802075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:02.803111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:02.803968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:02.803968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:02.803968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:02.803968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 255/519] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:02.804647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1dd386-980b-45d9-8616-af5ba1dd4da0 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:02.818652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558656d9-d8c6-49f4-85a9-55bace7423c0 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.855898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275040b-1f4c-4db5-8fa1-d3c1ae288203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:02.856524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275040b-1f4c-4db5-8fa1-d3c1ae288203 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:02.856807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275040b-1f4c-4db5-8fa1-d3c1ae288203 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:02.857069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275040b-1f4c-4db5-8fa1-d3c1ae288203 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:02.857772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275040b-1f4c-4db5-8fa1-d3c1ae288203 None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:02.858107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275040b-1f4c-4db5-8fa1-d3c1ae288203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:02.859462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275040b-1f4c-4db5-8fa1-d3c1ae288203 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:02.868449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 265/520] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:02.869336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275040b-1f4c-4db5-8fa1-d3c1ae288203 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:02.926864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1dd386-980b-45d9-8616-af5ba1dd4da0 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.936465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:02.966730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125681f3-710e-440c-be01-3182df79ffb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:02.967366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125681f3-710e-440c-be01-3182df79ffb2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:02.967649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125681f3-710e-440c-be01-3182df79ffb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:02.967923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125681f3-710e-440c-be01-3182df79ffb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:02.980397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125681f3-710e-440c-be01-3182df79ffb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:02.981170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275040b-1f4c-4db5-8fa1-d3c1ae288203 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:02.981878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125681f3-710e-440c-be01-3182df79ffb2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-125681f3-710e-440c-be01-3182df79ffb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:02.998589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:02.999543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:03.000375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:03.000375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:03.000375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:03.000375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 256/521] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:03.000583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125681f3-710e-440c-be01-3182df79ffb2 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.022203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ce83f-b6fc-4db8-98c7-d84786ef064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.022730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ce83f-b6fc-4db8-98c7-d84786ef064a None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.023004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ce83f-b6fc-4db8-98c7-d84786ef064a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.023270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ce83f-b6fc-4db8-98c7-d84786ef064a None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.024113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ce83f-b6fc-4db8-98c7-d84786ef064a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:03.024347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ce83f-b6fc-4db8-98c7-d84786ef064a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.025794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ce83f-b6fc-4db8-98c7-d84786ef064a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:03.038186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:03.049121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 266/522] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:03.049121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ce83f-b6fc-4db8-98c7-d84786ef064a None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.120386 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125681f3-710e-440c-be01-3182df79ffb2 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:03.150158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ce83f-b6fc-4db8-98c7-d84786ef064a None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:03.157060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f232156-ec79-4429-a2b8-1f091353b395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.157714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232156-ec79-4429-a2b8-1f091353b395 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.157997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232156-ec79-4429-a2b8-1f091353b395 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.158543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232156-ec79-4429-a2b8-1f091353b395 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.159033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f232156-ec79-4429-a2b8-1f091353b395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:03.159336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f232156-ec79-4429-a2b8-1f091353b395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.160701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f232156-ec79-4429-a2b8-1f091353b395 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:03.169618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:03.174063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 257/523] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:03.174906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f232156-ec79-4429-a2b8-1f091353b395 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.195540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee75729-8bc4-4394-8659-04c1f544101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.197127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee75729-8bc4-4394-8659-04c1f544101e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.197913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee75729-8bc4-4394-8659-04c1f544101e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.198335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee75729-8bc4-4394-8659-04c1f544101e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.211650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee75729-8bc4-4394-8659-04c1f544101e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.213320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee75729-8bc4-4394-8659-04c1f544101e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-4ee75729-8bc4-4394-8659-04c1f544101e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:03.229684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:03.230762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:03.231597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:03.231597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:03.231597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:03.234019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 267/524] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:03.235069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee75729-8bc4-4394-8659-04c1f544101e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.272565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:03.286413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f232156-ec79-4429-a2b8-1f091353b395 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:03.325883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bee14-92e4-4827-a5c2-db8e7732000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.326493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bee14-92e4-4827-a5c2-db8e7732000e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.326773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bee14-92e4-4827-a5c2-db8e7732000e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.327041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bee14-92e4-4827-a5c2-db8e7732000e None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.327692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bee14-92e4-4827-a5c2-db8e7732000e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:03.327996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bee14-92e4-4827-a5c2-db8e7732000e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.329360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bee14-92e4-4827-a5c2-db8e7732000e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:03.341441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 258/525] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:03.342272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bee14-92e4-4827-a5c2-db8e7732000e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.342932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee75729-8bc4-4394-8659-04c1f544101e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:03.381625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105c74a-f9a1-4dc4-af91-c6e6bda61a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.382238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105c74a-f9a1-4dc4-af91-c6e6bda61a21 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.382517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105c74a-f9a1-4dc4-af91-c6e6bda61a21 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.382793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105c74a-f9a1-4dc4-af91-c6e6bda61a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.383566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105c74a-f9a1-4dc4-af91-c6e6bda61a21 None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=1a4a95ae5ee84f259ffcd7528b0fa505) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:03.383876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105c74a-f9a1-4dc4-af91-c6e6bda61a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.385751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105c74a-f9a1-4dc4-af91-c6e6bda61a21 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:03.392847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 268/526] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:03 2022] GET /identity/v3/role_assignments?user.id=1a4a95ae5ee84f259ffcd7528b0fa505&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:03.393576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105c74a-f9a1-4dc4-af91-c6e6bda61a21 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.457531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:03.460045 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bee14-92e4-4827-a5c2-db8e7732000e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:03.510342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87041d5-9ddf-4ab7-967c-92838d4736a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.511362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87041d5-9ddf-4ab7-967c-92838d4736a7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.511876 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87041d5-9ddf-4ab7-967c-92838d4736a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.512353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87041d5-9ddf-4ab7-967c-92838d4736a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.520741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87041d5-9ddf-4ab7-967c-92838d4736a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.522674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87041d5-9ddf-4ab7-967c-92838d4736a7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-d87041d5-9ddf-4ab7-967c-92838d4736a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:03.536314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:03.537236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:03.538191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:03.538191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:03.538191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:03.538191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:03.538191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:03.538191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:03.538191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:03.538191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:03.540716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 259/527] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:03.541805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87041d5-9ddf-4ab7-967c-92838d4736a7 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.550427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105c74a-f9a1-4dc4-af91-c6e6bda61a21 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:03.559561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:03.606838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ccc16-2001-4a41-8ab6-62b0422e1eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.607753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ccc16-2001-4a41-8ab6-62b0422e1eb4 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.608178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ccc16-2001-4a41-8ab6-62b0422e1eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.608591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ccc16-2001-4a41-8ab6-62b0422e1eb4 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.610141 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ccc16-2001-4a41-8ab6-62b0422e1eb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:03.610629 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ccc16-2001-4a41-8ab6-62b0422e1eb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.612585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ccc16-2001-4a41-8ab6-62b0422e1eb4 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:03.635298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 269/528] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:03.636345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ccc16-2001-4a41-8ab6-62b0422e1eb4 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.672188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87041d5-9ddf-4ab7-967c-92838d4736a7 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:03.721840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4d46c-279d-48aa-9116-da9a56328215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.723166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4d46c-279d-48aa-9116-da9a56328215 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.723599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4d46c-279d-48aa-9116-da9a56328215 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.724151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4d46c-279d-48aa-9116-da9a56328215 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.733648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4d46c-279d-48aa-9116-da9a56328215 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.734650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ccc16-2001-4a41-8ab6-62b0422e1eb4 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:03.735484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4d46c-279d-48aa-9116-da9a56328215 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:03.736557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-7ce4d46c-279d-48aa-9116-da9a56328215 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:03.751255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:03.752339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:03.753446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:03.753446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:03.753446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:03.753446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:03.753446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:03.753446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:03.753446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:03.753446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:03.755888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 260/529] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:03.756818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4d46c-279d-48aa-9116-da9a56328215 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.784807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf457c2b-1bff-43e7-a9ce-3e15e56fd2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.786295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf457c2b-1bff-43e7-a9ce-3e15e56fd2dd None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.787096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf457c2b-1bff-43e7-a9ce-3e15e56fd2dd None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.787558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf457c2b-1bff-43e7-a9ce-3e15e56fd2dd None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.788817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf457c2b-1bff-43e7-a9ce-3e15e56fd2dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:03.789742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf457c2b-1bff-43e7-a9ce-3e15e56fd2dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.792597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf457c2b-1bff-43e7-a9ce-3e15e56fd2dd None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:03.809094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 270/530] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:03.810472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf457c2b-1bff-43e7-a9ce-3e15e56fd2dd None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.838887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:03.872835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4d46c-279d-48aa-9116-da9a56328215 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:03.909278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c25461-4733-4fb8-ad7a-c2f772f426fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.910093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c25461-4733-4fb8-ad7a-c2f772f426fe None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.910445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c25461-4733-4fb8-ad7a-c2f772f426fe None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.910817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c25461-4733-4fb8-ad7a-c2f772f426fe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.919694 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c25461-4733-4fb8-ad7a-c2f772f426fe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.921011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c25461-4733-4fb8-ad7a-c2f772f426fe None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-f4c25461-4733-4fb8-ad7a-c2f772f426fe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:03.932827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:03.933770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:03.934680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 261/531] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:03.935223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c25461-4733-4fb8-ad7a-c2f772f426fe None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:03.942739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf457c2b-1bff-43e7-a9ce-3e15e56fd2dd None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:03.984104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad98a6-ece0-47f2-b058-46a93367ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:03.985005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad98a6-ece0-47f2-b058-46a93367ec9d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:03.985554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad98a6-ece0-47f2-b058-46a93367ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:03.986019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad98a6-ece0-47f2-b058-46a93367ec9d None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:03.986896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad98a6-ece0-47f2-b058-46a93367ec9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:03.987412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad98a6-ece0-47f2-b058-46a93367ec9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:03.989033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad98a6-ece0-47f2-b058-46a93367ec9d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:04.004104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 271/532] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:04.005231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad98a6-ece0-47f2-b058-46a93367ec9d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:04.020935 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:04.052538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c25461-4733-4fb8-ad7a-c2f772f426fe None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:04.083325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:04.090098 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf6be3-9302-4853-9454-4f958d737f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:04.090749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf6be3-9302-4853-9454-4f958d737f9f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:04.091116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf6be3-9302-4853-9454-4f958d737f9f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:04.091397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf6be3-9302-4853-9454-4f958d737f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:04.092179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf6be3-9302-4853-9454-4f958d737f9f None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=dd04df68f9d84fc4bb981c8b69cc771a) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:04.092494 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf6be3-9302-4853-9454-4f958d737f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:04.094402 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf6be3-9302-4853-9454-4f958d737f9f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:04.100056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 262/533] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:03 2022] GET /identity/v3/role_assignments?user.id=dd04df68f9d84fc4bb981c8b69cc771a&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:04.101179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf6be3-9302-4853-9454-4f958d737f9f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:04.115337 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad98a6-ece0-47f2-b058-46a93367ec9d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:04.153855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38faae-45a8-4f9b-a43c-2ce8ba653244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:04.154994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38faae-45a8-4f9b-a43c-2ce8ba653244 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:04.155445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38faae-45a8-4f9b-a43c-2ce8ba653244 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:04.155879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38faae-45a8-4f9b-a43c-2ce8ba653244 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:04.156930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38faae-45a8-4f9b-a43c-2ce8ba653244 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:04.157813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38faae-45a8-4f9b-a43c-2ce8ba653244 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:04.159475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38faae-45a8-4f9b-a43c-2ce8ba653244 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:04.184173 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 272/534] 10.222.0.46 () {56 vars in 1139 bytes} [Mon Oct 3 14:39:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:04.192493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38faae-45a8-4f9b-a43c-2ce8ba653244 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:04.247706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf6be3-9302-4853-9454-4f958d737f9f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:04.324851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb750130-f73d-4611-8fea-4c942669c4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:04.326268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750130-f73d-4611-8fea-4c942669c4df None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:04.327091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750130-f73d-4611-8fea-4c942669c4df None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:04.327843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750130-f73d-4611-8fea-4c942669c4df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:04.328784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb750130-f73d-4611-8fea-4c942669c4df None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=64a9421bec3b4b9e89dec35d7556d025) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:04.329319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb750130-f73d-4611-8fea-4c942669c4df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:04.331021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb750130-f73d-4611-8fea-4c942669c4df None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:04.341604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 263/535] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:04 2022] GET /identity/v3/role_assignments?user.id=64a9421bec3b4b9e89dec35d7556d025&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:04.354117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:04.410288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38faae-45a8-4f9b-a43c-2ce8ba653244 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:04.459512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8c60c-a640-4ea8-a0a3-dec2c153072e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:04.460130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c60c-a640-4ea8-a0a3-dec2c153072e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:04.460408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c60c-a640-4ea8-a0a3-dec2c153072e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:04.460680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c60c-a640-4ea8-a0a3-dec2c153072e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:04.547299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8c60c-a640-4ea8-a0a3-dec2c153072e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:04.548783 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8c60c-a640-4ea8-a0a3-dec2c153072e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:04.619023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:04.650914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 273/536] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:39:04.656445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb750130-f73d-4611-8fea-4c942669c4df None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:04.758619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:04.807837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb750130-f73d-4611-8fea-4c942669c4df None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:04.853069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30378d-1f06-4649-9d19-92692f7398ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:04.853724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30378d-1f06-4649-9d19-92692f7398ae None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:04.854003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30378d-1f06-4649-9d19-92692f7398ae None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:04.854272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30378d-1f06-4649-9d19-92692f7398ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:04.871857 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30378d-1f06-4649-9d19-92692f7398ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:04.873247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30378d-1f06-4649-9d19-92692f7398ae None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-2d30378d-1f06-4649-9d19-92692f7398ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:04.891375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:04.892322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:04.893230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:04.893230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:04.893230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:04.897663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 264/537] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:04.902657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8c60c-a640-4ea8-a0a3-dec2c153072e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:05.040919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8c60c-a640-4ea8-a0a3-dec2c153072e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:05.075864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24ac07-2610-4c63-bddd-62e7cc8505bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:05.076472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24ac07-2610-4c63-bddd-62e7cc8505bb None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:05.076749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24ac07-2610-4c63-bddd-62e7cc8505bb None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:05.077015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24ac07-2610-4c63-bddd-62e7cc8505bb None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:05.077690 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24ac07-2610-4c63-bddd-62e7cc8505bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:05.077988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24ac07-2610-4c63-bddd-62e7cc8505bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:05.079345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24ac07-2610-4c63-bddd-62e7cc8505bb None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:05.091748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 274/538] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:05.102635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30378d-1f06-4649-9d19-92692f7398ae None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:05.114263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c88626-7f24-4073-973f-da11318046c5 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:05.114578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c88626-7f24-4073-973f-da11318046c5 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:05.115082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c88626-7f24-4073-973f-da11318046c5 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:05.116449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 275/539] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:05.126534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c59e-38c4-437f-8c8b-724cc0aea855 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:05.126717 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c59e-38c4-437f-8c8b-724cc0aea855 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:05.126992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c59e-38c4-437f-8c8b-724cc0aea855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:05.145469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e0f5c59e-38c4-437f-8c8b-724cc0aea855 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:05.146697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e0f5c59e-38c4-437f-8c8b-724cc0aea855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:05.184514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:05.256745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 276/540] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:05.263287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30378d-1f06-4649-9d19-92692f7398ae None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:05.283216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:05.307622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f592d-563a-4ff1-9249-3d9a6178e140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:05.308349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f592d-563a-4ff1-9249-3d9a6178e140 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:05.308631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f592d-563a-4ff1-9249-3d9a6178e140 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:05.308900 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f592d-563a-4ff1-9249-3d9a6178e140 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:05.315727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f592d-563a-4ff1-9249-3d9a6178e140 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:05.317167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f592d-563a-4ff1-9249-3d9a6178e140 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-7e7f592d-563a-4ff1-9249-3d9a6178e140 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:05.329160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:05.330417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:05.331247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:05.331247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:05.331247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:05.331247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:05.331247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:05.331247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:05.331247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:05.331247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:05.331247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 265/541] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:05.338369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c59e-38c4-437f-8c8b-724cc0aea855 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:05.460022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c59e-38c4-437f-8c8b-724cc0aea855 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:05.504850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bbea0-bd6d-4369-b34b-557ad0976c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:05.505720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bbea0-bd6d-4369-b34b-557ad0976c79 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:05.506157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bbea0-bd6d-4369-b34b-557ad0976c79 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:05.506607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bbea0-bd6d-4369-b34b-557ad0976c79 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:05.507442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bbea0-bd6d-4369-b34b-557ad0976c79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:05.507970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bbea0-bd6d-4369-b34b-557ad0976c79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:05.509705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bbea0-bd6d-4369-b34b-557ad0976c79 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:05.527055 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 277/542] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:05.533017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f592d-563a-4ff1-9249-3d9a6178e140 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:05.600022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bbea0-bd6d-4369-b34b-557ad0976c79 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:05.626993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:05.697327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f592d-563a-4ff1-9249-3d9a6178e140 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:05.707540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bbea0-bd6d-4369-b34b-557ad0976c79 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:05.743410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e007b-a25e-41fb-a672-d9dc600caa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:05.743954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e007b-a25e-41fb-a672-d9dc600caa81 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:05.744236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e007b-a25e-41fb-a672-d9dc600caa81 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:05.744506 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e007b-a25e-41fb-a672-d9dc600caa81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:05.758062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c192129-7d3a-4cf7-b5f1-2e7357291fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:05.758670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c192129-7d3a-4cf7-b5f1-2e7357291fdd None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:05.758950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c192129-7d3a-4cf7-b5f1-2e7357291fdd None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:05.759222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c192129-7d3a-4cf7-b5f1-2e7357291fdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:05.759952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c192129-7d3a-4cf7-b5f1-2e7357291fdd None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=05e9009410a54ce3ab88fa0cac87a7f9) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:05.760259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c192129-7d3a-4cf7-b5f1-2e7357291fdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:05.761736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c192129-7d3a-4cf7-b5f1-2e7357291fdd None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:05.767425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e007b-a25e-41fb-a672-d9dc600caa81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-f34e007b-a25e-41fb-a672-d9dc600caa81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:05.772643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:05.773591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:05.774785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 278/543] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:05.779696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e007b-a25e-41fb-a672-d9dc600caa81 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:05.782050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 266/544] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:05 2022] GET /identity/v3/role_assignments?user.id=05e9009410a54ce3ab88fa0cac87a7f9&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:05.869749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:05.891014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e007b-a25e-41fb-a672-d9dc600caa81 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:05.927360 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411b810-5b16-4534-9635-d0d7a19db43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:05.927987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b810-5b16-4534-9635-d0d7a19db43d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:05.928265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b810-5b16-4534-9635-d0d7a19db43d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:05.928536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b810-5b16-4534-9635-d0d7a19db43d None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:05.929225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411b810-5b16-4534-9635-d0d7a19db43d None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:05.929556 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411b810-5b16-4534-9635-d0d7a19db43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:05.931018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411b810-5b16-4534-9635-d0d7a19db43d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:05.940474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 279/545] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:05.946782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c192129-7d3a-4cf7-b5f1-2e7357291fdd None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:06.027949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:06.100945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c192129-7d3a-4cf7-b5f1-2e7357291fdd None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:06.149447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b5912-6fca-4f6a-ae59-6b3779ccbc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:06.150156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5912-6fca-4f6a-ae59-6b3779ccbc2f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:06.150438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5912-6fca-4f6a-ae59-6b3779ccbc2f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:06.150706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b5912-6fca-4f6a-ae59-6b3779ccbc2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:06.165985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b5912-6fca-4f6a-ae59-6b3779ccbc2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:06.167480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b5912-6fca-4f6a-ae59-6b3779ccbc2f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-cb3b5912-6fca-4f6a-ae59-6b3779ccbc2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:06.187158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:06.188125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:06.189194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:06.189194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:06.189194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:06.189194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 267/546] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:06.194188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411b810-5b16-4534-9635-d0d7a19db43d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:06.330751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411b810-5b16-4534-9635-d0d7a19db43d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:06.366859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac5cc1-6bff-4dd2-b04b-5572f3b8e521 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:06.367888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac5cc1-6bff-4dd2-b04b-5572f3b8e521 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:06.367959 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac5cc1-6bff-4dd2-b04b-5572f3b8e521 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:06.369236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 268/547] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:06.374793 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da50648-4fd9-4a6c-9d6a-ce9c69eb15ad None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:06.375072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da50648-4fd9-4a6c-9d6a-ce9c69eb15ad None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:06.375343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da50648-4fd9-4a6c-9d6a-ce9c69eb15ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:06.389559 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1524d3-febc-40d0-af34-4566c2b6d332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:06.390179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1524d3-febc-40d0-af34-4566c2b6d332 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:06.390510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1524d3-febc-40d0-af34-4566c2b6d332 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:06.390773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1524d3-febc-40d0-af34-4566c2b6d332 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:06.391453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1524d3-febc-40d0-af34-4566c2b6d332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:06.391780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1524d3-febc-40d0-af34-4566c2b6d332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:06.393257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1524d3-febc-40d0-af34-4566c2b6d332 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:06.401550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-5da50648-4fd9-4a6c-9d6a-ce9c69eb15ad None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:06.402758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5da50648-4fd9-4a6c-9d6a-ce9c69eb15ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:06.419706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 280/548] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:06.424821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1524d3-febc-40d0-af34-4566c2b6d332 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:06.430106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:06.558143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 269/549] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:39:06.581980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1524d3-febc-40d0-af34-4566c2b6d332 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:06.631869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b1982-3f23-43f4-a361-2a471fccac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:06.632508 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b1982-3f23-43f4-a361-2a471fccac96 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:06.633322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b1982-3f23-43f4-a361-2a471fccac96 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:06.633626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b1982-3f23-43f4-a361-2a471fccac96 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:06.644085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0154e-2633-466f-ad66-32e2cbef112b None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:06.644680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0154e-2633-466f-ad66-32e2cbef112b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:06.645143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0154e-2633-466f-ad66-32e2cbef112b None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:06.646976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 270/550] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:06.649102 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b1982-3f23-43f4-a361-2a471fccac96 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:06.650593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b1982-3f23-43f4-a361-2a471fccac96 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:06.653632 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9c542-ad6f-4c8d-8058-34369a74fa31 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:06.654234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9c542-ad6f-4c8d-8058-34369a74fa31 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:06.654702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9c542-ad6f-4c8d-8058-34369a74fa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-3a2b1982-3f23-43f4-a361-2a471fccac96 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:06.668577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:06.669723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:06.671234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 281/551] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:06.677545 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b1982-3f23-43f4-a361-2a471fccac96 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:06.685191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-c3b9c542-ad6f-4c8d-8058-34369a74fa31 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:06.687978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c3b9c542-ad6f-4c8d-8058-34369a74fa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:06.731427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:06.769885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:06.808482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b1982-3f23-43f4-a361-2a471fccac96 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:06.848282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 271/552] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:39:06.861376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6333217-f6cb-403d-9da6-d6f5ca2e5ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:06.862243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6333217-f6cb-403d-9da6-d6f5ca2e5ef9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:06.862682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6333217-f6cb-403d-9da6-d6f5ca2e5ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:06.863096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6333217-f6cb-403d-9da6-d6f5ca2e5ef9 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:06.863971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6333217-f6cb-403d-9da6-d6f5ca2e5ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:06.864420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6333217-f6cb-403d-9da6-d6f5ca2e5ef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:06.866410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6333217-f6cb-403d-9da6-d6f5ca2e5ef9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:06.890752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 282/553] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 3 14:39:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:06.896605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9c542-ad6f-4c8d-8058-34369a74fa31 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:06.935842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6333217-f6cb-403d-9da6-d6f5ca2e5ef9 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:07.042709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9c542-ad6f-4c8d-8058-34369a74fa31 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:07.048347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6333217-f6cb-403d-9da6-d6f5ca2e5ef9 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:07.072967 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:07.089641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ee350-0c84-43f2-8e67-94299d3ed63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:07.092987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ee350-0c84-43f2-8e67-94299d3ed63a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.093264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ee350-0c84-43f2-8e67-94299d3ed63a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.093553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ee350-0c84-43f2-8e67-94299d3ed63a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.095765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509536e6-0bd9-46e0-adf2-c1bccfdac230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:07.096427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509536e6-0bd9-46e0-adf2-c1bccfdac230 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.096802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509536e6-0bd9-46e0-adf2-c1bccfdac230 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.097073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509536e6-0bd9-46e0-adf2-c1bccfdac230 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.107915 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509536e6-0bd9-46e0-adf2-c1bccfdac230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-509536e6-0bd9-46e0-adf2-c1bccfdac230 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:07.112789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:07.113711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:07.114524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 283/554] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:07.115458 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441f17f-bfc9-4d69-8661-07d76b38cdf2 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.116153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441f17f-bfc9-4d69-8661-07d76b38cdf2 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.116153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441f17f-bfc9-4d69-8661-07d76b38cdf2 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.117909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 284/555] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:07.123242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb12027-fd36-423a-bfef-c0d3b668a903 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.123362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:07.123985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb12027-fd36-423a-bfef-c0d3b668a903 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.123985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb12027-fd36-423a-bfef-c0d3b668a903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.141477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-2fb12027-fd36-423a-bfef-c0d3b668a903 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:07.142692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2fb12027-fd36-423a-bfef-c0d3b668a903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:07.180277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ee350-0c84-43f2-8e67-94299d3ed63a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:07.181757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ee350-0c84-43f2-8e67-94299d3ed63a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:07.217586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 285/556] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:07.218479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb12027-fd36-423a-bfef-c0d3b668a903 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:07.311988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 272/557] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:39:07.312852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ee350-0c84-43f2-8e67-94299d3ed63a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:07.345934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb12027-fd36-423a-bfef-c0d3b668a903 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:07.360297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:07.364853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:07.385525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c752b-deac-409d-894f-743df571d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:07.386223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c752b-deac-409d-894f-743df571d57c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.386503 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c752b-deac-409d-894f-743df571d57c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.386771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c752b-deac-409d-894f-743df571d57c None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.387433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c752b-deac-409d-894f-743df571d57c None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:07.387730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c752b-deac-409d-894f-743df571d57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:07.389191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c752b-deac-409d-894f-743df571d57c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:07.404585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 286/558] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:07.405465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c752b-deac-409d-894f-743df571d57c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:07.425442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ee350-0c84-43f2-8e67-94299d3ed63a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:07.467491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033372b-0bc6-4c9a-8885-0b7f90469692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:07.468125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033372b-0bc6-4c9a-8885-0b7f90469692 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.468406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033372b-0bc6-4c9a-8885-0b7f90469692 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.468679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033372b-0bc6-4c9a-8885-0b7f90469692 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.481840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033372b-0bc6-4c9a-8885-0b7f90469692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-4033372b-0bc6-4c9a-8885-0b7f90469692 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:07.486735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:07.487630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:07.488426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:07.493090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 273/559] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:07.493979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033372b-0bc6-4c9a-8885-0b7f90469692 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:07.513671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c752b-deac-409d-894f-743df571d57c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:07.546885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bcf32-339b-448f-bd9e-fafea54d02ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:07.547512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bcf32-339b-448f-bd9e-fafea54d02ed None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.547788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bcf32-339b-448f-bd9e-fafea54d02ed None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.548056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bcf32-339b-448f-bd9e-fafea54d02ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.559137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bcf32-339b-448f-bd9e-fafea54d02ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:07.560598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bcf32-339b-448f-bd9e-fafea54d02ed None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-666bcf32-339b-448f-bd9e-fafea54d02ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:07.574812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:07.575718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:07.576588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:07.576588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:07.576588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:07.576588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 287/560] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:07.576805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bcf32-339b-448f-bd9e-fafea54d02ed None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:07.605390 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:07.610230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033372b-0bc6-4c9a-8885-0b7f90469692 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:07.620892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:07.647672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe754d6-df26-4e86-a7bd-99ed1e33cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:07.649011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe754d6-df26-4e86-a7bd-99ed1e33cceb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.649291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe754d6-df26-4e86-a7bd-99ed1e33cceb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.649586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe754d6-df26-4e86-a7bd-99ed1e33cceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.660217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe754d6-df26-4e86-a7bd-99ed1e33cceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:07.661687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe754d6-df26-4e86-a7bd-99ed1e33cceb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-efe754d6-df26-4e86-a7bd-99ed1e33cceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:07.676096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:07.677181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:07.678065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:07.678065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:07.678065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:07.678065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 274/561] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:07.678065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe754d6-df26-4e86-a7bd-99ed1e33cceb None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:07.691191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bcf32-339b-448f-bd9e-fafea54d02ed None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:07.728176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d51c0-2bb5-44a8-8d26-0281864cde45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:07.728781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d51c0-2bb5-44a8-8d26-0281864cde45 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.729053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d51c0-2bb5-44a8-8d26-0281864cde45 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.729331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d51c0-2bb5-44a8-8d26-0281864cde45 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.730033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d51c0-2bb5-44a8-8d26-0281864cde45 None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:07.730368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d51c0-2bb5-44a8-8d26-0281864cde45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:07.731750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d51c0-2bb5-44a8-8d26-0281864cde45 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:07.745802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 288/562] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:07.746635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d51c0-2bb5-44a8-8d26-0281864cde45 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:07.783033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe754d6-df26-4e86-a7bd-99ed1e33cceb None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:07.818725 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497fd5-993b-421b-b4a1-7160c7f1314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:07.819359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497fd5-993b-421b-b4a1-7160c7f1314f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.819637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497fd5-993b-421b-b4a1-7160c7f1314f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.819962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497fd5-993b-421b-b4a1-7160c7f1314f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.831652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26497fd5-993b-421b-b4a1-7160c7f1314f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:07.834898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-26497fd5-993b-421b-b4a1-7160c7f1314f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:07.839671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:07.840687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:07.841518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:07.845748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 275/563] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:07.846596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497fd5-993b-421b-b4a1-7160c7f1314f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:07.853027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d51c0-2bb5-44a8-8d26-0281864cde45 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:07.871497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:07.894884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98839cc-f5f0-4b43-b25a-442257555f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:07.895658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98839cc-f5f0-4b43-b25a-442257555f25 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.896302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98839cc-f5f0-4b43-b25a-442257555f25 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.896585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98839cc-f5f0-4b43-b25a-442257555f25 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.897516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98839cc-f5f0-4b43-b25a-442257555f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:07.897842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98839cc-f5f0-4b43-b25a-442257555f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:07.899740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98839cc-f5f0-4b43-b25a-442257555f25 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:07.920225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 289/564] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:07.920225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98839cc-f5f0-4b43-b25a-442257555f25 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:07.952017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497fd5-993b-421b-b4a1-7160c7f1314f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:07.988297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09906ee-e303-42f6-a6aa-a23cdf5c03f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:07.989026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09906ee-e303-42f6-a6aa-a23cdf5c03f9 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:07.989319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09906ee-e303-42f6-a6aa-a23cdf5c03f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:07.990019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09906ee-e303-42f6-a6aa-a23cdf5c03f9 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:07.990529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09906ee-e303-42f6-a6aa-a23cdf5c03f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:07.990840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09906ee-e303-42f6-a6aa-a23cdf5c03f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:07.992264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09906ee-e303-42f6-a6aa-a23cdf5c03f9 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.010765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 276/565] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:08.010765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09906ee-e303-42f6-a6aa-a23cdf5c03f9 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.028954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98839cc-f5f0-4b43-b25a-442257555f25 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.062637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e09eb-adc6-4169-a4b3-006cf1808400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.063256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e09eb-adc6-4169-a4b3-006cf1808400 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.063528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e09eb-adc6-4169-a4b3-006cf1808400 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.063792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e09eb-adc6-4169-a4b3-006cf1808400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.074605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e09eb-adc6-4169-a4b3-006cf1808400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.076042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e09eb-adc6-4169-a4b3-006cf1808400 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.085234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-d79e09eb-adc6-4169-a4b3-006cf1808400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:08.091467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:08.092651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:08.093599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:08.093599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:08.093599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:08.093599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 290/566] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:08.094137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e09eb-adc6-4169-a4b3-006cf1808400 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.103234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:08.124863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09906ee-e303-42f6-a6aa-a23cdf5c03f9 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.162856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e8ab8-82d9-420e-920f-c730ed1abb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.163808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e8ab8-82d9-420e-920f-c730ed1abb72 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.164352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e8ab8-82d9-420e-920f-c730ed1abb72 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.164956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e8ab8-82d9-420e-920f-c730ed1abb72 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.165993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e8ab8-82d9-420e-920f-c730ed1abb72 None admin] RBAC: Adding query filter params (name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:08.166381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e8ab8-82d9-420e-920f-c730ed1abb72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.167763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e8ab8-82d9-420e-920f-c730ed1abb72 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.176533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 277/567] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:08.177261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e8ab8-82d9-420e-920f-c730ed1abb72 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.200698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e09eb-adc6-4169-a4b3-006cf1808400 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.239305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d792df7-8b39-4bc7-bb78-1e43e47818e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.240212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d792df7-8b39-4bc7-bb78-1e43e47818e5 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.240492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d792df7-8b39-4bc7-bb78-1e43e47818e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.240760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d792df7-8b39-4bc7-bb78-1e43e47818e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.248487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d792df7-8b39-4bc7-bb78-1e43e47818e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.250038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d792df7-8b39-4bc7-bb78-1e43e47818e5 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-1d792df7-8b39-4bc7-bb78-1e43e47818e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:08.263843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:08.264815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:08.265720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:08.265720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:08.265720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:08.265720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:08.265720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:08.265720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:08.265720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:08.265720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:08.269860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 291/568] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:08.270741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d792df7-8b39-4bc7-bb78-1e43e47818e5 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.291636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e8ab8-82d9-420e-920f-c730ed1abb72 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.328472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e741f4-e54c-48a8-9eae-e62ef929ad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.329124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e741f4-e54c-48a8-9eae-e62ef929ad17 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.329399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e741f4-e54c-48a8-9eae-e62ef929ad17 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.329693 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e741f4-e54c-48a8-9eae-e62ef929ad17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.338092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e741f4-e54c-48a8-9eae-e62ef929ad17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.339516 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e741f4-e54c-48a8-9eae-e62ef929ad17 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.341813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-29e741f4-e54c-48a8-9eae-e62ef929ad17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:08.350937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:08.352030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:08.352964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 278/569] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:08.353580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e741f4-e54c-48a8-9eae-e62ef929ad17 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.356178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:08.395017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d792df7-8b39-4bc7-bb78-1e43e47818e5 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.427426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11147a8c-303d-46e8-b54b-a37327cf4fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.428031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11147a8c-303d-46e8-b54b-a37327cf4fa1 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.428308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11147a8c-303d-46e8-b54b-a37327cf4fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.428580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11147a8c-303d-46e8-b54b-a37327cf4fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.429217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11147a8c-303d-46e8-b54b-a37327cf4fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:08.430042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11147a8c-303d-46e8-b54b-a37327cf4fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.431431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11147a8c-303d-46e8-b54b-a37327cf4fa1 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.443106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 292/570] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:08.444079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11147a8c-303d-46e8-b54b-a37327cf4fa1 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.463660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e741f4-e54c-48a8-9eae-e62ef929ad17 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.496483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e226d-0e28-427e-8cb9-daeb29c0e997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.497116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e226d-0e28-427e-8cb9-daeb29c0e997 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.497399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e226d-0e28-427e-8cb9-daeb29c0e997 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.497853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e226d-0e28-427e-8cb9-daeb29c0e997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.508420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e226d-0e28-427e-8cb9-daeb29c0e997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.509930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e226d-0e28-427e-8cb9-daeb29c0e997 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-502e226d-0e28-427e-8cb9-daeb29c0e997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:08.525066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:08.526015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:08.526889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:08.526889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:08.526889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:08.526889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 279/571] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:08.527658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b9d9d-9af4-4cfc-a555-c1948a48fd93 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.527658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b9d9d-9af4-4cfc-a555-c1948a48fd93 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.527824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b9d9d-9af4-4cfc-a555-c1948a48fd93 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.529144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 280/572] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:08.529815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b9d9d-9af4-4cfc-a555-c1948a48fd93 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.544487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11147a8c-303d-46e8-b54b-a37327cf4fa1 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.564637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:08.578284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae994be-a0ae-4b29-9107-e43ed439f2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.578890 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae994be-a0ae-4b29-9107-e43ed439f2b8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.579171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae994be-a0ae-4b29-9107-e43ed439f2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.579439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae994be-a0ae-4b29-9107-e43ed439f2b8 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.580131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae994be-a0ae-4b29-9107-e43ed439f2b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:08.580438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae994be-a0ae-4b29-9107-e43ed439f2b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.581905 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae994be-a0ae-4b29-9107-e43ed439f2b8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.586383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:08.593749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 293/573] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:08.594550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae994be-a0ae-4b29-9107-e43ed439f2b8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.636655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b9d9d-9af4-4cfc-a555-c1948a48fd93 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.669859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e531895-837f-4b38-851f-6834b5e28734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.670457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531895-837f-4b38-851f-6834b5e28734 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.670738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531895-837f-4b38-851f-6834b5e28734 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.671018 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531895-837f-4b38-851f-6834b5e28734 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.671746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531895-837f-4b38-851f-6834b5e28734 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:08.672054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531895-837f-4b38-851f-6834b5e28734 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.673484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531895-837f-4b38-851f-6834b5e28734 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.692033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae994be-a0ae-4b29-9107-e43ed439f2b8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.693815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 281/574] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:08.694732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e531895-837f-4b38-851f-6834b5e28734 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.726036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c123cc9-4569-407b-a16a-aa7bc97e80e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.726689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c123cc9-4569-407b-a16a-aa7bc97e80e7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.726962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c123cc9-4569-407b-a16a-aa7bc97e80e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.727237 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c123cc9-4569-407b-a16a-aa7bc97e80e7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.734781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c123cc9-4569-407b-a16a-aa7bc97e80e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.736228 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c123cc9-4569-407b-a16a-aa7bc97e80e7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-7c123cc9-4569-407b-a16a-aa7bc97e80e7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:08.747841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:08.748817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 294/575] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc1313-8fc2-4c05-a1ad-8d431658b9d3 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.754535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc1313-8fc2-4c05-a1ad-8d431658b9d3 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.755209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc1313-8fc2-4c05-a1ad-8d431658b9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.768174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-a3bc1313-8fc2-4c05-a1ad-8d431658b9d3 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:08.769374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a3bc1313-8fc2-4c05-a1ad-8d431658b9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:08.790458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e531895-837f-4b38-851f-6834b5e28734 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.802546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:08.803100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:08.825663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823e8b5-5340-4e95-bac2-d5b2fe0d7a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.826266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823e8b5-5340-4e95-bac2-d5b2fe0d7a24 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.826549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823e8b5-5340-4e95-bac2-d5b2fe0d7a24 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.826829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823e8b5-5340-4e95-bac2-d5b2fe0d7a24 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.827473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823e8b5-5340-4e95-bac2-d5b2fe0d7a24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:08.827779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823e8b5-5340-4e95-bac2-d5b2fe0d7a24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.829134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823e8b5-5340-4e95-bac2-d5b2fe0d7a24 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.835008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 295/576] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:08.836361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc1313-8fc2-4c05-a1ad-8d431658b9d3 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.840798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 282/577] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:08.841921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823e8b5-5340-4e95-bac2-d5b2fe0d7a24 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.935734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc1313-8fc2-4c05-a1ad-8d431658b9d3 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.947490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823e8b5-5340-4e95-bac2-d5b2fe0d7a24 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:08.970374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02c9fc-3081-44bf-954c-6b9c34ccb425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.970971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02c9fc-3081-44bf-954c-6b9c34ccb425 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.971248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02c9fc-3081-44bf-954c-6b9c34ccb425 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.971523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02c9fc-3081-44bf-954c-6b9c34ccb425 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:08.972247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02c9fc-3081-44bf-954c-6b9c34ccb425 None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=6fbf23ae20ee4f6c979e256d2d4daa5f) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:08.972552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02c9fc-3081-44bf-954c-6b9c34ccb425 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:08.974018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02c9fc-3081-44bf-954c-6b9c34ccb425 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:08.989773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 296/578] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/role_assignments?user.id=6fbf23ae20ee4f6c979e256d2d4daa5f&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:08.989773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02c9fc-3081-44bf-954c-6b9c34ccb425 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:08.993256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172278f-3111-4445-b8cb-d3993034c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:08.993937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172278f-3111-4445-b8cb-d3993034c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:08.994221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172278f-3111-4445-b8cb-d3993034c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:08.994491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172278f-3111-4445-b8cb-d3993034c9a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.014470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172278f-3111-4445-b8cb-d3993034c9a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:09.016178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172278f-3111-4445-b8cb-d3993034c9a1 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-a172278f-3111-4445-b8cb-d3993034c9a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:09.034881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:09.036060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:09.037124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:09.037124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:09.037124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:09.041439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 283/579] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:09.043043 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172278f-3111-4445-b8cb-d3993034c9a1 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:09.045815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:09.088087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:09.135074 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02c9fc-3081-44bf-954c-6b9c34ccb425 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:09.176371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e409ff1-c0b6-4d67-9b69-2646f755cb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:09.176990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e409ff1-c0b6-4d67-9b69-2646f755cb7b None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:09.177271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e409ff1-c0b6-4d67-9b69-2646f755cb7b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:09.177617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e409ff1-c0b6-4d67-9b69-2646f755cb7b None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.178365 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e409ff1-c0b6-4d67-9b69-2646f755cb7b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:09.178756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e409ff1-c0b6-4d67-9b69-2646f755cb7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:09.180680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e409ff1-c0b6-4d67-9b69-2646f755cb7b None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:09.188237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172278f-3111-4445-b8cb-d3993034c9a1 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:09.203027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 297/580] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 3 14:39:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:09.203027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e409ff1-c0b6-4d67-9b69-2646f755cb7b None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:09.228286 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae53eb-7883-4edd-81fc-6fe702762b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:09.228926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae53eb-7883-4edd-81fc-6fe702762b9d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:09.229210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae53eb-7883-4edd-81fc-6fe702762b9d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:09.229484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae53eb-7883-4edd-81fc-6fe702762b9d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.237315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae53eb-7883-4edd-81fc-6fe702762b9d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:09.239839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae53eb-7883-4edd-81fc-6fe702762b9d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-ebae53eb-7883-4edd-81fc-6fe702762b9d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:09.252656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:09.253807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 284/581] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:09.254836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae53eb-7883-4edd-81fc-6fe702762b9d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:09.311405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:09.321653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e409ff1-c0b6-4d67-9b69-2646f755cb7b None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:09.348445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:09.359286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d07dcd-d9a4-40cc-9d2f-1cac8b3061cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:09.359996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d07dcd-d9a4-40cc-9d2f-1cac8b3061cf None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:09.360272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d07dcd-d9a4-40cc-9d2f-1cac8b3061cf None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:09.360551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d07dcd-d9a4-40cc-9d2f-1cac8b3061cf None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.361197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d07dcd-d9a4-40cc-9d2f-1cac8b3061cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:09.361518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d07dcd-d9a4-40cc-9d2f-1cac8b3061cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:09.362898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d07dcd-d9a4-40cc-9d2f-1cac8b3061cf None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:09.368185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae53eb-7883-4edd-81fc-6fe702762b9d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:09.379536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 298/582] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:09.379536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d07dcd-d9a4-40cc-9d2f-1cac8b3061cf None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:09.406042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ff786-b36b-4bc7-9f50-f110bd2d54f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:09.406678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ff786-b36b-4bc7-9f50-f110bd2d54f3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:09.406963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ff786-b36b-4bc7-9f50-f110bd2d54f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:09.407235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ff786-b36b-4bc7-9f50-f110bd2d54f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.419108 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ff786-b36b-4bc7-9f50-f110bd2d54f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-3d9ff786-b36b-4bc7-9f50-f110bd2d54f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:09.426232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:09.427169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:09.427989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 285/583] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:09.428669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ff786-b36b-4bc7-9f50-f110bd2d54f3 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:09.494718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d07dcd-d9a4-40cc-9d2f-1cac8b3061cf None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:09.536520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6d578-b7b3-464d-bef1-0a3ed27a86d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:09.536855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ff786-b36b-4bc7-9f50-f110bd2d54f3 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:09.537281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6d578-b7b3-464d-bef1-0a3ed27a86d9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:09.537467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6d578-b7b3-464d-bef1-0a3ed27a86d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:09.537714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6d578-b7b3-464d-bef1-0a3ed27a86d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.549051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6d578-b7b3-464d-bef1-0a3ed27a86d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:09.550291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6d578-b7b3-464d-bef1-0a3ed27a86d9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-e2d6d578-b7b3-464d-bef1-0a3ed27a86d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:09.564642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:09.566128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:09.567019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:09.567019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:09.567019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:09.567019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 299/584] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:09.567019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6d578-b7b3-464d-bef1-0a3ed27a86d9 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:09.571932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:09.578503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75baf72-42e0-4878-8f5f-80224d1cf3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:09.579115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75baf72-42e0-4878-8f5f-80224d1cf3db None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:09.579400 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75baf72-42e0-4878-8f5f-80224d1cf3db None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:09.579673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75baf72-42e0-4878-8f5f-80224d1cf3db None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.580405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75baf72-42e0-4878-8f5f-80224d1cf3db None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:09.580722 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75baf72-42e0-4878-8f5f-80224d1cf3db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:09.582183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75baf72-42e0-4878-8f5f-80224d1cf3db None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:09.605159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 286/585] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:09.605159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75baf72-42e0-4878-8f5f-80224d1cf3db None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:09.605159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:09.676861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6d578-b7b3-464d-bef1-0a3ed27a86d9 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:09.712662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75baf72-42e0-4878-8f5f-80224d1cf3db None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:09.713482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6763798-1f9d-45dc-a7b2-e7c9eaf5b3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:09.714143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6763798-1f9d-45dc-a7b2-e7c9eaf5b3d9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:09.714425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6763798-1f9d-45dc-a7b2-e7c9eaf5b3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:09.714699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6763798-1f9d-45dc-a7b2-e7c9eaf5b3d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.721468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6763798-1f9d-45dc-a7b2-e7c9eaf5b3d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:09.722941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6763798-1f9d-45dc-a7b2-e7c9eaf5b3d9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-f6763798-1f9d-45dc-a7b2-e7c9eaf5b3d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:09.734722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:09.735621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:09.736500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:09.736500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:09.736500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:09.736500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:09.736500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:09.736500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:09.736500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:09.736500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:09.736500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 300/586] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:09.736980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6763798-1f9d-45dc-a7b2-e7c9eaf5b3d9 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:09.759078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58eac8a-ae18-4b17-85d4-eb07f888dda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:09.760147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eac8a-ae18-4b17-85d4-eb07f888dda7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:09.760572 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eac8a-ae18-4b17-85d4-eb07f888dda7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:09.760985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eac8a-ae18-4b17-85d4-eb07f888dda7 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.761860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58eac8a-ae18-4b17-85d4-eb07f888dda7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:09.762724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58eac8a-ae18-4b17-85d4-eb07f888dda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:09.764509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58eac8a-ae18-4b17-85d4-eb07f888dda7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:09.776282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 287/587] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:09.777256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58eac8a-ae18-4b17-85d4-eb07f888dda7 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:09.803803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:09.844008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:09.844725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6763798-1f9d-45dc-a7b2-e7c9eaf5b3d9 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:09.883347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676b22a-a178-404b-a787-a14df7eb17a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:09.883946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676b22a-a178-404b-a787-a14df7eb17a8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:09.884218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676b22a-a178-404b-a787-a14df7eb17a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:09.884490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676b22a-a178-404b-a787-a14df7eb17a8 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.885123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676b22a-a178-404b-a787-a14df7eb17a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:09.885424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676b22a-a178-404b-a787-a14df7eb17a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:09.886818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676b22a-a178-404b-a787-a14df7eb17a8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:09.888265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58eac8a-ae18-4b17-85d4-eb07f888dda7 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:09.903930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 301/588] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:09.903930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676b22a-a178-404b-a787-a14df7eb17a8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:09.927660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2c729-a9dd-453f-a0a9-9d50286211a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:09.928338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2c729-a9dd-453f-a0a9-9d50286211a8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:09.928615 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2c729-a9dd-453f-a0a9-9d50286211a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:09.928882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2c729-a9dd-453f-a0a9-9d50286211a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:09.940169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2c729-a9dd-453f-a0a9-9d50286211a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:09.941618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2c729-a9dd-453f-a0a9-9d50286211a8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-19b2c729-a9dd-453f-a0a9-9d50286211a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:09.955988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:09.957009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:09.958269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:09.958269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:09.958269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:09.958269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 288/589] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:09.958269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2c729-a9dd-453f-a0a9-9d50286211a8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.016124 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676b22a-a178-404b-a787-a14df7eb17a8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:10.038932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:10.050980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f72ca-4ca4-4a58-bf67-7ae638b67488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:10.051579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f72ca-4ca4-4a58-bf67-7ae638b67488 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:10.051853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f72ca-4ca4-4a58-bf67-7ae638b67488 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:10.052117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f72ca-4ca4-4a58-bf67-7ae638b67488 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:10.052799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f72ca-4ca4-4a58-bf67-7ae638b67488 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:10.053099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f72ca-4ca4-4a58-bf67-7ae638b67488 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:10.055135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f72ca-4ca4-4a58-bf67-7ae638b67488 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:10.071992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 302/590] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:10.071992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f72ca-4ca4-4a58-bf67-7ae638b67488 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.072212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2c729-a9dd-453f-a0a9-9d50286211a8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:10.089688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:10.108047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6248f5-d0c7-4799-b12c-31d4bfa6811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:10.108679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6248f5-d0c7-4799-b12c-31d4bfa6811c None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:10.108961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6248f5-d0c7-4799-b12c-31d4bfa6811c None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:10.109236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6248f5-d0c7-4799-b12c-31d4bfa6811c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:10.120435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6248f5-d0c7-4799-b12c-31d4bfa6811c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:10.121917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6248f5-d0c7-4799-b12c-31d4bfa6811c None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-3d6248f5-d0c7-4799-b12c-31d4bfa6811c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:10.136568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:10.137489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:10.138379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:10.138379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:10.138379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:10.139794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 289/591] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:10.140692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6248f5-d0c7-4799-b12c-31d4bfa6811c None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.179173 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f72ca-4ca4-4a58-bf67-7ae638b67488 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:10.214369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12aea1-4f11-4638-89e9-ea9e0b945596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:10.215003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12aea1-4f11-4638-89e9-ea9e0b945596 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:10.215282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12aea1-4f11-4638-89e9-ea9e0b945596 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:10.215549 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12aea1-4f11-4638-89e9-ea9e0b945596 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:10.222127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12aea1-4f11-4638-89e9-ea9e0b945596 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:10.223572 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12aea1-4f11-4638-89e9-ea9e0b945596 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-db12aea1-4f11-4638-89e9-ea9e0b945596 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:10.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:10.236550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:10.237422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:10.237422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:10.237422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:10.237422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:10.237422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:10.237422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:10.237422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:10.237422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:10.237422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 303/592] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:10.238705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12aea1-4f11-4638-89e9-ea9e0b945596 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.257010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6248f5-d0c7-4799-b12c-31d4bfa6811c None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:10.293417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f449d-1432-4971-8c67-960e7829b1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:10.293979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f449d-1432-4971-8c67-960e7829b1fc None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:10.294366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f449d-1432-4971-8c67-960e7829b1fc None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:10.294536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f449d-1432-4971-8c67-960e7829b1fc None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:10.295154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f449d-1432-4971-8c67-960e7829b1fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:10.295458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f449d-1432-4971-8c67-960e7829b1fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:10.296822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f449d-1432-4971-8c67-960e7829b1fc None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:10.297698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:10.309342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 290/593] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:10.309995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f449d-1432-4971-8c67-960e7829b1fc None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.339062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:10.360771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12aea1-4f11-4638-89e9-ea9e0b945596 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:10.394914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cde66-f8da-4c97-96a6-ee28f8ac426a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:10.395580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cde66-f8da-4c97-96a6-ee28f8ac426a None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:10.395858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cde66-f8da-4c97-96a6-ee28f8ac426a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:10.396122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cde66-f8da-4c97-96a6-ee28f8ac426a None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/1a4a95ae5ee84f259ffcd7528b0fa505/roles/f5822988729a45539e07519e59cf7bc4` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:10.411531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cde66-f8da-4c97-96a6-ee28f8ac426a None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=1a4a95ae5ee84f259ffcd7528b0fa505, role_id=f5822988729a45539e07519e59cf7bc4)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:10.413022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cde66-f8da-4c97-96a6-ee28f8ac426a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:10.415303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f449d-1432-4971-8c67-960e7829b1fc None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:10.426069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 304/594] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:39:10 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/1a4a95ae5ee84f259ffcd7528b0fa505/roles/f5822988729a45539e07519e59cf7bc4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:39:10.427113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cde66-f8da-4c97-96a6-ee28f8ac426a None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.465904 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c8838-4962-4671-bd9c-01a6ee0e3454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:10.466537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c8838-4962-4671-bd9c-01a6ee0e3454 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:10.466814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c8838-4962-4671-bd9c-01a6ee0e3454 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:10.467084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c8838-4962-4671-bd9c-01a6ee0e3454 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:10.467746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c8838-4962-4671-bd9c-01a6ee0e3454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:10.468046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c8838-4962-4671-bd9c-01a6ee0e3454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:10.469529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c8838-4962-4671-bd9c-01a6ee0e3454 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:10.490137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 291/595] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:10.491165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c8838-4962-4671-bd9c-01a6ee0e3454 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.594270 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:10.613841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:10.622533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c8838-4962-4671-bd9c-01a6ee0e3454 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:10.628015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cde66-f8da-4c97-96a6-ee28f8ac426a None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:10.665062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c03f81-e302-41e9-b2c9-80078d6243bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:10.665752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c03f81-e302-41e9-b2c9-80078d6243bf None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:10.666091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c03f81-e302-41e9-b2c9-80078d6243bf None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:10.666296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c03f81-e302-41e9-b2c9-80078d6243bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:10.670632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7735984-6bf7-4e04-bf6e-b5b3d74308b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:10.671506 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7735984-6bf7-4e04-bf6e-b5b3d74308b0 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:10.671986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7735984-6bf7-4e04-bf6e-b5b3d74308b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:10.672484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7735984-6bf7-4e04-bf6e-b5b3d74308b0 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:10.673662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7735984-6bf7-4e04-bf6e-b5b3d74308b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:10.673782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c03f81-e302-41e9-b2c9-80078d6243bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:10.674303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7735984-6bf7-4e04-bf6e-b5b3d74308b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:10.675162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c03f81-e302-41e9-b2c9-80078d6243bf None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:10.676135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7735984-6bf7-4e04-bf6e-b5b3d74308b0 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-14c03f81-e302-41e9-b2c9-80078d6243bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:10.686769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:10.687710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:10.688562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:10.688562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:10.688562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:10.688562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:10.688562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:10.688562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:10.688562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:10.688562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:10.688562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 292/596] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:10.689029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c03f81-e302-41e9-b2c9-80078d6243bf None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.692796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 305/597] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:10.698906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7735984-6bf7-4e04-bf6e-b5b3d74308b0 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.817737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c03f81-e302-41e9-b2c9-80078d6243bf None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:10.820533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7735984-6bf7-4e04-bf6e-b5b3d74308b0 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:10.860484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60d8f8-d9e3-4cc0-93ae-2a2755901e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:10.861521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60d8f8-d9e3-4cc0-93ae-2a2755901e70 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:10.862032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60d8f8-d9e3-4cc0-93ae-2a2755901e70 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:10.862501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60d8f8-d9e3-4cc0-93ae-2a2755901e70 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:10.863033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0ff06-89d6-43f3-8471-8601330606f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:10.863852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0ff06-89d6-43f3-8471-8601330606f1 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:10.864117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0ff06-89d6-43f3-8471-8601330606f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:10.864387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0ff06-89d6-43f3-8471-8601330606f1 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:10.865373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0ff06-89d6-43f3-8471-8601330606f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:10.865714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0ff06-89d6-43f3-8471-8601330606f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:10.868159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0ff06-89d6-43f3-8471-8601330606f1 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:10.872607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60d8f8-d9e3-4cc0-93ae-2a2755901e70 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:10.874285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60d8f8-d9e3-4cc0-93ae-2a2755901e70 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-2d60d8f8-d9e3-4cc0-93ae-2a2755901e70 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:10.886006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:10.886944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:10.887845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:10.887845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:10.887845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:10.887845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:10.887845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:10.887845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:10.887845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:10.887845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:10.887845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 14:39:10.887845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:10.888355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 306/598] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:10.888355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0ff06-89d6-43f3-8471-8601330606f1 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.890873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 293/599] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:10.892768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60d8f8-d9e3-4cc0-93ae-2a2755901e70 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:10.905669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:10.913445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:10.994510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0ff06-89d6-43f3-8471-8601330606f1 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:11.019306 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60d8f8-d9e3-4cc0-93ae-2a2755901e70 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:11.048963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1510eae-b934-4920-a1f1-06bc612fd352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:11.049664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510eae-b934-4920-a1f1-06bc612fd352 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:11.049940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510eae-b934-4920-a1f1-06bc612fd352 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:11.050209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510eae-b934-4920-a1f1-06bc612fd352 None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/dd04df68f9d84fc4bb981c8b69cc771a/roles/f5822988729a45539e07519e59cf7bc4` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:11.070735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1510eae-b934-4920-a1f1-06bc612fd352 None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=dd04df68f9d84fc4bb981c8b69cc771a, role_id=f5822988729a45539e07519e59cf7bc4)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:11.072471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1510eae-b934-4920-a1f1-06bc612fd352 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:11.074151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a07c1-67c9-4d45-8a76-06f433673d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:11.075223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a07c1-67c9-4d45-8a76-06f433673d3f None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:11.075502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a07c1-67c9-4d45-8a76-06f433673d3f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:11.076170 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a07c1-67c9-4d45-8a76-06f433673d3f None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/64a9421bec3b4b9e89dec35d7556d025/roles/f5822988729a45539e07519e59cf7bc4` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:11.088036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 307/600] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:39:10 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/dd04df68f9d84fc4bb981c8b69cc771a/roles/f5822988729a45539e07519e59cf7bc4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:39:11.088959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1510eae-b934-4920-a1f1-06bc612fd352 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:11.099652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a07c1-67c9-4d45-8a76-06f433673d3f None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=64a9421bec3b4b9e89dec35d7556d025, role_id=f5822988729a45539e07519e59cf7bc4)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:11.101336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a07c1-67c9-4d45-8a76-06f433673d3f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:11.120976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 294/601] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:39:10 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/64a9421bec3b4b9e89dec35d7556d025/roles/f5822988729a45539e07519e59cf7bc4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:39:11.146503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c5e1e-4a7c-4dae-8bb4-f965cc0774e3 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:11.146842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c5e1e-4a7c-4dae-8bb4-f965cc0774e3 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:11.147062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c5e1e-4a7c-4dae-8bb4-f965cc0774e3 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:11.149331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 295/602] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:11.154941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8217d5-87e5-44dd-a13d-f9e62df99196 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:11.155251 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8217d5-87e5-44dd-a13d-f9e62df99196 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:11.155501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8217d5-87e5-44dd-a13d-f9e62df99196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:11.174777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-8b8217d5-87e5-44dd-a13d-f9e62df99196 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:11.175435 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:11.176621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8b8217d5-87e5-44dd-a13d-f9e62df99196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:11.222792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1510eae-b934-4920-a1f1-06bc612fd352 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:11.261927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:11.285094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242cd35-84d0-40e9-981c-666b8c394acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:11.285828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242cd35-84d0-40e9-981c-666b8c394acd None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:11.286103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242cd35-84d0-40e9-981c-666b8c394acd None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:11.286372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242cd35-84d0-40e9-981c-666b8c394acd None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:11.287122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242cd35-84d0-40e9-981c-666b8c394acd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:11.287425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242cd35-84d0-40e9-981c-666b8c394acd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:11.290637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242cd35-84d0-40e9-981c-666b8c394acd None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:11.294008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 296/603] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:39:11.321594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 308/604] 10.222.0.46 () {56 vars in 1139 bytes} [Mon Oct 3 14:39:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:11.327681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8217d5-87e5-44dd-a13d-f9e62df99196 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:11.464924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8217d5-87e5-44dd-a13d-f9e62df99196 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:11.500721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56316a-b225-4241-b06c-c3387d7d7279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:11.501383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56316a-b225-4241-b06c-c3387d7d7279 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:11.501688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56316a-b225-4241-b06c-c3387d7d7279 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:11.501983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56316a-b225-4241-b06c-c3387d7d7279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:11.513311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56316a-b225-4241-b06c-c3387d7d7279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:11.514777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56316a-b225-4241-b06c-c3387d7d7279 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-fb56316a-b225-4241-b06c-c3387d7d7279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:11.530197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:11.531130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:11.531996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:11.531996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:11.531996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:11.531996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 297/605] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:11.538215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242cd35-84d0-40e9-981c-666b8c394acd None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:11.584682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56316a-b225-4241-b06c-c3387d7d7279 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:11.652977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:11.711355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242cd35-84d0-40e9-981c-666b8c394acd None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:11.751417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:11.756709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56316a-b225-4241-b06c-c3387d7d7279 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:11.769189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ca9df-95d8-499b-bd6a-60fb9097b8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:11.769858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ca9df-95d8-499b-bd6a-60fb9097b8b0 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:11.770141 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ca9df-95d8-499b-bd6a-60fb9097b8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:11.770415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ca9df-95d8-499b-bd6a-60fb9097b8b0 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:11.771067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ca9df-95d8-499b-bd6a-60fb9097b8b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:11.771371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ca9df-95d8-499b-bd6a-60fb9097b8b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:11.772746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ca9df-95d8-499b-bd6a-60fb9097b8b0 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:11.784809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 309/606] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:11.794494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ca9df-95d8-499b-bd6a-60fb9097b8b0 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:11.820639 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f23d13-fdb8-48c2-93b3-ae5ddd7f9a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:11.822154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f23d13-fdb8-48c2-93b3-ae5ddd7f9a92 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:11.822645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f23d13-fdb8-48c2-93b3-ae5ddd7f9a92 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:11.823132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f23d13-fdb8-48c2-93b3-ae5ddd7f9a92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:11.836231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f23d13-fdb8-48c2-93b3-ae5ddd7f9a92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-59f23d13-fdb8-48c2-93b3-ae5ddd7f9a92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:11.841833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:11.842759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:11.843583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:11.846228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 298/607] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:11.852687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f23d13-fdb8-48c2-93b3-ae5ddd7f9a92 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:11.955321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:11.959881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f23d13-fdb8-48c2-93b3-ae5ddd7f9a92 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:11.996656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbcedd-7444-429a-921a-257a85f1c445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:11.997343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbcedd-7444-429a-921a-257a85f1c445 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:11.997723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbcedd-7444-429a-921a-257a85f1c445 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:11.997995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbcedd-7444-429a-921a-257a85f1c445 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:11.998682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbcedd-7444-429a-921a-257a85f1c445 None admin] RBAC: Adding query filter params (name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:11.999033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbcedd-7444-429a-921a-257a85f1c445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:12.000472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbcedd-7444-429a-921a-257a85f1c445 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:12.010122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 299/608] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:12.015705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbcedd-7444-429a-921a-257a85f1c445 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:12.032096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ca9df-95d8-499b-bd6a-60fb9097b8b0 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:12.118226 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74179-cf34-4098-a31d-96331c52b896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:12.118871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74179-cf34-4098-a31d-96331c52b896 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:12.119152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74179-cf34-4098-a31d-96331c52b896 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:12.119414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74179-cf34-4098-a31d-96331c52b896 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:12.137990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b74179-cf34-4098-a31d-96331c52b896 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:12.140586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b74179-cf34-4098-a31d-96331c52b896 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-37b74179-cf34-4098-a31d-96331c52b896 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:12.168024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:12.168957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:12.169880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:12.169880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:12.169880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:12.169880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:12.169880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:12.169880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:12.169880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:12.169880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:12.173528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 310/609] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:12.176218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74179-cf34-4098-a31d-96331c52b896 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:12.178473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbcedd-7444-429a-921a-257a85f1c445 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:12.244709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85507394-1e60-4d25-a935-77b9c58fd38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:12.245391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507394-1e60-4d25-a935-77b9c58fd38a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:12.245692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507394-1e60-4d25-a935-77b9c58fd38a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:12.245962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507394-1e60-4d25-a935-77b9c58fd38a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:12.255772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:12.266919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85507394-1e60-4d25-a935-77b9c58fd38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:12.269419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85507394-1e60-4d25-a935-77b9c58fd38a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-85507394-1e60-4d25-a935-77b9c58fd38a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:12.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:12.297342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:12.298275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:12.298275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:12.298275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:12.302456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 300/610] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:12.306157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85507394-1e60-4d25-a935-77b9c58fd38a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:12.320751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:12.375447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74179-cf34-4098-a31d-96331c52b896 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:12.439732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f211d3-e193-4759-b922-38017f815dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:12.440816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f211d3-e193-4759-b922-38017f815dad None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:12.441890 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f211d3-e193-4759-b922-38017f815dad None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:12.442179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f211d3-e193-4759-b922-38017f815dad None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:12.442895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f211d3-e193-4759-b922-38017f815dad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:12.443205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f211d3-e193-4759-b922-38017f815dad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:12.444655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f211d3-e193-4759-b922-38017f815dad None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:12.459294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85507394-1e60-4d25-a935-77b9c58fd38a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:12.467306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 311/611] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:12.476685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f211d3-e193-4759-b922-38017f815dad None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:12.520791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319cc95-5e1a-4794-badf-9cff9a4da33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:12.521692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319cc95-5e1a-4794-badf-9cff9a4da33c None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:12.522168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319cc95-5e1a-4794-badf-9cff9a4da33c None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:12.523025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319cc95-5e1a-4794-badf-9cff9a4da33c None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:12.524451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319cc95-5e1a-4794-badf-9cff9a4da33c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:12.525145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319cc95-5e1a-4794-badf-9cff9a4da33c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:12.528049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319cc95-5e1a-4794-badf-9cff9a4da33c None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:12.557380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 301/612] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:12.559768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319cc95-5e1a-4794-badf-9cff9a4da33c None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:12.653114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f211d3-e193-4759-b922-38017f815dad None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:12.655050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:12.688009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:12.714200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0da905-2bf8-4714-a5af-d67e0681520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:12.714872 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0da905-2bf8-4714-a5af-d67e0681520e None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:12.715151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0da905-2bf8-4714-a5af-d67e0681520e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:12.715425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0da905-2bf8-4714-a5af-d67e0681520e None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/05e9009410a54ce3ab88fa0cac87a7f9/roles/f5822988729a45539e07519e59cf7bc4` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:12.719297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319cc95-5e1a-4794-badf-9cff9a4da33c None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:12.746826 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0da905-2bf8-4714-a5af-d67e0681520e None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=05e9009410a54ce3ab88fa0cac87a7f9, role_id=f5822988729a45539e07519e59cf7bc4)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:12.748285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0da905-2bf8-4714-a5af-d67e0681520e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:12.760270 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d7f5a-8ae4-4de0-bee2-8596bf8f6d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:12.761340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d7f5a-8ae4-4de0-bee2-8596bf8f6d47 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:12.761940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d7f5a-8ae4-4de0-bee2-8596bf8f6d47 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:12.762492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d7f5a-8ae4-4de0-bee2-8596bf8f6d47 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:12.771042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d7f5a-8ae4-4de0-bee2-8596bf8f6d47 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:12.772803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d7f5a-8ae4-4de0-bee2-8596bf8f6d47 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:12.776103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 312/613] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:39:12 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/05e9009410a54ce3ab88fa0cac87a7f9/roles/f5822988729a45539e07519e59cf7bc4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-d91d7f5a-8ae4-4de0-bee2-8596bf8f6d47 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:12.786569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:12.787591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:12.788522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:12.788522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:12.788522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:12.788522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:12.788522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:12.788522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:12.788522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:12.788522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:12.788522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:12.788522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:12.795834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 302/614] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:12.798065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0da905-2bf8-4714-a5af-d67e0681520e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:12.927897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0da905-2bf8-4714-a5af-d67e0681520e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:12.973719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b881e3-c954-42f9-9b32-140e55d2ebbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:12.974724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b881e3-c954-42f9-9b32-140e55d2ebbf None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:12.975213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b881e3-c954-42f9-9b32-140e55d2ebbf None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:12.975687 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b881e3-c954-42f9-9b32-140e55d2ebbf None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:12.976692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b881e3-c954-42f9-9b32-140e55d2ebbf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:12.977197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b881e3-c954-42f9-9b32-140e55d2ebbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:12.978859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b881e3-c954-42f9-9b32-140e55d2ebbf None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:13.000153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 313/615] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 3 14:39:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:13.006442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d7f5a-8ae4-4de0-bee2-8596bf8f6d47 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:13.117490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d7f5a-8ae4-4de0-bee2-8596bf8f6d47 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:13.159293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484203f-15f9-4513-a0a6-7555f3679264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:13.159935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484203f-15f9-4513-a0a6-7555f3679264 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:13.160213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484203f-15f9-4513-a0a6-7555f3679264 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:13.160482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484203f-15f9-4513-a0a6-7555f3679264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:13.168826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:13.173177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484203f-15f9-4513-a0a6-7555f3679264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:13.174688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484203f-15f9-4513-a0a6-7555f3679264 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-1484203f-15f9-4513-a0a6-7555f3679264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:13.193290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:13.194350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:13.196153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:13.196153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:13.196153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:13.196153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 303/616] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:13.202210 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b881e3-c954-42f9-9b32-140e55d2ebbf None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:13.213086 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dfd22-5fdd-48b6-a255-f617fb07bbe3 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:13.215145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dfd22-5fdd-48b6-a255-f617fb07bbe3 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:13.216641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dfd22-5fdd-48b6-a255-f617fb07bbe3 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:13.216837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:13.219048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 304/617] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:13.224390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4184be-f063-4d05-9a64-f52837203b02 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:13.224754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4184be-f063-4d05-9a64-f52837203b02 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:13.225093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4184be-f063-4d05-9a64-f52837203b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:13.261446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-aa4184be-f063-4d05-9a64-f52837203b02 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:13.267471 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-aa4184be-f063-4d05-9a64-f52837203b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:13.387482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 305/618] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:39:13.404478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b881e3-c954-42f9-9b32-140e55d2ebbf None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:13.451967 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853c193-bb6e-47e9-9726-7df9e07ca467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:13.452664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853c193-bb6e-47e9-9726-7df9e07ca467 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:13.453011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853c193-bb6e-47e9-9726-7df9e07ca467 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:13.453345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853c193-bb6e-47e9-9726-7df9e07ca467 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:13.454084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853c193-bb6e-47e9-9726-7df9e07ca467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:13.454454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853c193-bb6e-47e9-9726-7df9e07ca467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:13.455883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853c193-bb6e-47e9-9726-7df9e07ca467 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:13.468879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 314/619] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:13.474452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4184be-f063-4d05-9a64-f52837203b02 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:13.594776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4184be-f063-4d05-9a64-f52837203b02 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:13.619714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:13.643149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83e8db-17bd-4259-b474-4e69b204cb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:13.643720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83e8db-17bd-4259-b474-4e69b204cb2f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:13.644059 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83e8db-17bd-4259-b474-4e69b204cb2f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:13.644312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83e8db-17bd-4259-b474-4e69b204cb2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:13.657018 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83e8db-17bd-4259-b474-4e69b204cb2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:13.658640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83e8db-17bd-4259-b474-4e69b204cb2f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-5f83e8db-17bd-4259-b474-4e69b204cb2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:13.675698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:13.676678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:13.677850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:13.677850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:13.677850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:13.677850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:13.677850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:13.677850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:13.677850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:13.677850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:13.677850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 306/620] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:13.683305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853c193-bb6e-47e9-9726-7df9e07ca467 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:13.735150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83e8db-17bd-4259-b474-4e69b204cb2f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:13.771016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:13.809112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853c193-bb6e-47e9-9726-7df9e07ca467 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:13.846218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627c590-f17c-42bd-aeac-b31dd7e72c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:13.847208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627c590-f17c-42bd-aeac-b31dd7e72c04 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:13.847675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627c590-f17c-42bd-aeac-b31dd7e72c04 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:13.848191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627c590-f17c-42bd-aeac-b31dd7e72c04 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:13.849056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627c590-f17c-42bd-aeac-b31dd7e72c04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:13.849586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627c590-f17c-42bd-aeac-b31dd7e72c04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:13.852024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627c590-f17c-42bd-aeac-b31dd7e72c04 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:13.865761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 315/621] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:13.867262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029375f-c51e-4a70-90f0-7db4e71e8869 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:13.867546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029375f-c51e-4a70-90f0-7db4e71e8869 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:13.867819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029375f-c51e-4a70-90f0-7db4e71e8869 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:13.869263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 316/622] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:13.874022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83e8db-17bd-4259-b474-4e69b204cb2f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:13.876017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fedad-3edc-445c-994e-9ddcb34c7de3 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:13.876301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fedad-3edc-445c-994e-9ddcb34c7de3 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:13.876577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fedad-3edc-445c-994e-9ddcb34c7de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:13.895314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-a77fedad-3edc-445c-994e-9ddcb34c7de3 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:13.896444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a77fedad-3edc-445c-994e-9ddcb34c7de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:13.917787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552c75e-e599-4c98-b0ef-1fc92941f36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:13.918795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552c75e-e599-4c98-b0ef-1fc92941f36c None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:13.919801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552c75e-e599-4c98-b0ef-1fc92941f36c None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:13.920295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552c75e-e599-4c98-b0ef-1fc92941f36c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:13.932730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552c75e-e599-4c98-b0ef-1fc92941f36c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-8552c75e-e599-4c98-b0ef-1fc92941f36c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:13.938510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:13.939405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:13.940257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 307/623] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:13.941529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552c75e-e599-4c98-b0ef-1fc92941f36c None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:13.967773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 317/624] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:13.969327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66040bdf-98fe-4a30-8f92-a97a66ea7482 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:13.969639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66040bdf-98fe-4a30-8f92-a97a66ea7482 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:13.969912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66040bdf-98fe-4a30-8f92-a97a66ea7482 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:13.971282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 318/625] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:13.971819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66040bdf-98fe-4a30-8f92-a97a66ea7482 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:13.987596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:14.041472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:14.076307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552c75e-e599-4c98-b0ef-1fc92941f36c None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:14.084819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66040bdf-98fe-4a30-8f92-a97a66ea7482 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:14.119087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200ca88-bb91-445b-9a25-b476a2809da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:14.119990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200ca88-bb91-445b-9a25-b476a2809da4 None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:14.120433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200ca88-bb91-445b-9a25-b476a2809da4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:14.120794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93f8ea-8030-4fa6-bffb-2d43bb88d3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:14.121042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200ca88-bb91-445b-9a25-b476a2809da4 None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/6fbf23ae20ee4f6c979e256d2d4daa5f/roles/f5822988729a45539e07519e59cf7bc4` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:14.121561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93f8ea-8030-4fa6-bffb-2d43bb88d3d7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:14.121909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93f8ea-8030-4fa6-bffb-2d43bb88d3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:14.122248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93f8ea-8030-4fa6-bffb-2d43bb88d3d7 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:14.122981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93f8ea-8030-4fa6-bffb-2d43bb88d3d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:14.123354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93f8ea-8030-4fa6-bffb-2d43bb88d3d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:14.124858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93f8ea-8030-4fa6-bffb-2d43bb88d3d7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:14.138773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 319/626] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:14.140175 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200ca88-bb91-445b-9a25-b476a2809da4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=6fbf23ae20ee4f6c979e256d2d4daa5f, role_id=f5822988729a45539e07519e59cf7bc4)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:14.140308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0153f-1dbb-4271-af86-be43a6a22907 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:14.140626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0153f-1dbb-4271-af86-be43a6a22907 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:14.141004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0153f-1dbb-4271-af86-be43a6a22907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:14.142239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200ca88-bb91-445b-9a25-b476a2809da4 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:14.156300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 308/627] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:39:13 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/6fbf23ae20ee4f6c979e256d2d4daa5f/roles/f5822988729a45539e07519e59cf7bc4 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:39:14.157357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200ca88-bb91-445b-9a25-b476a2809da4 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:14.160578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-96a0153f-1dbb-4271-af86-be43a6a22907 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:14.162318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-96a0153f-1dbb-4271-af86-be43a6a22907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:14.243274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 320/628] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:14.269176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:14.303337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200ca88-bb91-445b-9a25-b476a2809da4 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:14.333945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0153f-1dbb-4271-af86-be43a6a22907 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:14.381717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d654c3-d417-48d5-b49a-8b8a92d4288e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:14.382447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d654c3-d417-48d5-b49a-8b8a92d4288e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:14.382724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d654c3-d417-48d5-b49a-8b8a92d4288e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:14.382995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d654c3-d417-48d5-b49a-8b8a92d4288e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:14.399429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:14.412271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d654c3-d417-48d5-b49a-8b8a92d4288e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:14.413726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d654c3-d417-48d5-b49a-8b8a92d4288e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-62d654c3-d417-48d5-b49a-8b8a92d4288e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:14.428848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:14.430612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:14.431553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:14.431553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:14.431553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:14.431730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 309/629] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:14.442736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d654c3-d417-48d5-b49a-8b8a92d4288e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:14.466700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0153f-1dbb-4271-af86-be43a6a22907 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:14.529574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91b1c0-c557-466d-9066-27c1814bb360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:14.530206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91b1c0-c557-466d-9066-27c1814bb360 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:14.530477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91b1c0-c557-466d-9066-27c1814bb360 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:14.530739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91b1c0-c557-466d-9066-27c1814bb360 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:14.547208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91b1c0-c557-466d-9066-27c1814bb360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-5f91b1c0-c557-466d-9066-27c1814bb360 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:14.551992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:14.552978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:14.553857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 321/630] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:14.561022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91b1c0-c557-466d-9066-27c1814bb360 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:14.608756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d654c3-d417-48d5-b49a-8b8a92d4288e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:14.648541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe5725-03c8-441c-8a13-ab31b1a99e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:14.649620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe5725-03c8-441c-8a13-ab31b1a99e87 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:14.650117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe5725-03c8-441c-8a13-ab31b1a99e87 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:14.650621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe5725-03c8-441c-8a13-ab31b1a99e87 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:14.651482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe5725-03c8-441c-8a13-ab31b1a99e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:14.652017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe5725-03c8-441c-8a13-ab31b1a99e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:14.654152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe5725-03c8-441c-8a13-ab31b1a99e87 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:14.663078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:14.671826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 310/631] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:14.673371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe5725-03c8-441c-8a13-ab31b1a99e87 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:14.678933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91b1c0-c557-466d-9066-27c1814bb360 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:14.692395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:14.716989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cece62-ac7f-4da2-b669-8515d16faf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:14.717873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cece62-ac7f-4da2-b669-8515d16faf8e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:14.718313 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cece62-ac7f-4da2-b669-8515d16faf8e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:14.718730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cece62-ac7f-4da2-b669-8515d16faf8e None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:14.719535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cece62-ac7f-4da2-b669-8515d16faf8e None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:14.720133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cece62-ac7f-4da2-b669-8515d16faf8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:14.722209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cece62-ac7f-4da2-b669-8515d16faf8e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:14.734998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 322/632] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:14.736129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cece62-ac7f-4da2-b669-8515d16faf8e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:14.795676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe5725-03c8-441c-8a13-ab31b1a99e87 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:14.834373 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2828e-a232-4551-b1c3-b8de638e03ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:14.835404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2828e-a232-4551-b1c3-b8de638e03ed None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:14.835934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2828e-a232-4551-b1c3-b8de638e03ed None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:14.836408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2828e-a232-4551-b1c3-b8de638e03ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:14.848851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2828e-a232-4551-b1c3-b8de638e03ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:14.851515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cece62-ac7f-4da2-b669-8515d16faf8e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-40f2828e-a232-4551-b1c3-b8de638e03ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:14.854913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:14.855933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:14.856758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:14.859295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 311/633] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:14.860435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2828e-a232-4551-b1c3-b8de638e03ed None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:14.896481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803fc11-fa6b-44c7-84e6-8741723b61a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:14.897271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803fc11-fa6b-44c7-84e6-8741723b61a8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:14.897657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803fc11-fa6b-44c7-84e6-8741723b61a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:14.898064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803fc11-fa6b-44c7-84e6-8741723b61a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:14.907499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803fc11-fa6b-44c7-84e6-8741723b61a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:14.909746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803fc11-fa6b-44c7-84e6-8741723b61a8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-4803fc11-fa6b-44c7-84e6-8741723b61a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:14.925177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:14.926253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:14.927653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 323/634] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 3 14:39:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:14.930179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803fc11-fa6b-44c7-84e6-8741723b61a8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:14.931548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:14.977301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:14.982940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2828e-a232-4551-b1c3-b8de638e03ed None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:15.026374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c3889-3b28-446c-98d0-221643c75d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:15.027018 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c3889-3b28-446c-98d0-221643c75d6e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.027297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c3889-3b28-446c-98d0-221643c75d6e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.027568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c3889-3b28-446c-98d0-221643c75d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.039192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c3889-3b28-446c-98d0-221643c75d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:15.040653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c3889-3b28-446c-98d0-221643c75d6e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-129c3889-3b28-446c-98d0-221643c75d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:15.055563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:15.056555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:15.057408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:15.057408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:15.057408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:15.061248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 312/635] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:15.062287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c3889-3b28-446c-98d0-221643c75d6e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:15.070737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803fc11-fa6b-44c7-84e6-8741723b61a8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:15.110101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e61f2-f49b-4429-ab98-67dd7bd29a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:15.110710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e61f2-f49b-4429-ab98-67dd7bd29a9d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.111037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e61f2-f49b-4429-ab98-67dd7bd29a9d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.111306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e61f2-f49b-4429-ab98-67dd7bd29a9d None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.111964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e61f2-f49b-4429-ab98-67dd7bd29a9d None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:15.112265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e61f2-f49b-4429-ab98-67dd7bd29a9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:15.113761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e61f2-f49b-4429-ab98-67dd7bd29a9d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:15.123231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 324/636] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:15.124145 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e61f2-f49b-4429-ab98-67dd7bd29a9d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:15.192929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c3889-3b28-446c-98d0-221643c75d6e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:15.204286 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:15.229934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc506d7-376c-4c2c-8235-b5349dcbd152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:15.230543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc506d7-376c-4c2c-8235-b5349dcbd152 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.230827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc506d7-376c-4c2c-8235-b5349dcbd152 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.231099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc506d7-376c-4c2c-8235-b5349dcbd152 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.231838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc506d7-376c-4c2c-8235-b5349dcbd152 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:15.232122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc506d7-376c-4c2c-8235-b5349dcbd152 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:15.233569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc506d7-376c-4c2c-8235-b5349dcbd152 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:15.238867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:15.248128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e61f2-f49b-4429-ab98-67dd7bd29a9d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:15.257691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 313/637] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 3 14:39:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:15.258582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc506d7-376c-4c2c-8235-b5349dcbd152 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:15.289607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac3fc5-394d-4787-925c-d7433b29e475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:15.290636 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac3fc5-394d-4787-925c-d7433b29e475 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.291122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac3fc5-394d-4787-925c-d7433b29e475 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.291587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac3fc5-394d-4787-925c-d7433b29e475 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.292463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac3fc5-394d-4787-925c-d7433b29e475 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:15.293070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac3fc5-394d-4787-925c-d7433b29e475 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:15.294959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac3fc5-394d-4787-925c-d7433b29e475 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:15.310322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 325/638] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:15.311554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac3fc5-394d-4787-925c-d7433b29e475 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:15.390143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc506d7-376c-4c2c-8235-b5349dcbd152 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:15.425886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd5ffe-cdc4-4000-a3b7-267480dc0939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:15.426513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd5ffe-cdc4-4000-a3b7-267480dc0939 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.426790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd5ffe-cdc4-4000-a3b7-267480dc0939 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.427062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd5ffe-cdc4-4000-a3b7-267480dc0939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.433090 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac3fc5-394d-4787-925c-d7433b29e475 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:15.437212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd5ffe-cdc4-4000-a3b7-267480dc0939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:15.438684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd5ffe-cdc4-4000-a3b7-267480dc0939 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-f9fd5ffe-cdc4-4000-a3b7-267480dc0939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:15.455555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:15.456455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:15.457357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:15.457357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:15.457357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:15.457357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 314/639] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:15.463301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd5ffe-cdc4-4000-a3b7-267480dc0939 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:15.476390 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4465211-a4a7-4f76-8203-4d85bebdd772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:15.476999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4465211-a4a7-4f76-8203-4d85bebdd772 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.477277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4465211-a4a7-4f76-8203-4d85bebdd772 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.477581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4465211-a4a7-4f76-8203-4d85bebdd772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.481969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:15.505706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:15.550937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4465211-a4a7-4f76-8203-4d85bebdd772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:15.552534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4465211-a4a7-4f76-8203-4d85bebdd772 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:15.567504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd5ffe-cdc4-4000-a3b7-267480dc0939 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:15.603739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e8aa5-747d-495d-9b58-72592cac1591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:15.604369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e8aa5-747d-495d-9b58-72592cac1591 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.604645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e8aa5-747d-495d-9b58-72592cac1591 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.604921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e8aa5-747d-495d-9b58-72592cac1591 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.612423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e8aa5-747d-495d-9b58-72592cac1591 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:15.613940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e8aa5-747d-495d-9b58-72592cac1591 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-913e8aa5-747d-495d-9b58-72592cac1591 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:15.625295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:15.626524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 315/640] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:15.627687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee09023-1099-4b2a-af64-4fa2365f0cfe None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.628498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee09023-1099-4b2a-af64-4fa2365f0cfe None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.628498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee09023-1099-4b2a-af64-4fa2365f0cfe None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.634208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 316/641] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:15.634208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee09023-1099-4b2a-af64-4fa2365f0cfe None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:15.671341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 326/642] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:39:15.672792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43930ba5-6ad3-4c57-bd7a-ff8c1a0de31b None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.673213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43930ba5-6ad3-4c57-bd7a-ff8c1a0de31b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.673644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43930ba5-6ad3-4c57-bd7a-ff8c1a0de31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.691847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-43930ba5-6ad3-4c57-bd7a-ff8c1a0de31b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:15.693023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-43930ba5-6ad3-4c57-bd7a-ff8c1a0de31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:15.719737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:15.741830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee09023-1099-4b2a-af64-4fa2365f0cfe None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:15.746020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:15.773524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 327/643] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:15.773524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43930ba5-6ad3-4c57-bd7a-ff8c1a0de31b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:15.789308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469c2c2-034c-454a-b0a0-bc553855f31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:15.790138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469c2c2-034c-454a-b0a0-bc553855f31d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.790517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469c2c2-034c-454a-b0a0-bc553855f31d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.790883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469c2c2-034c-454a-b0a0-bc553855f31d None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.791741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469c2c2-034c-454a-b0a0-bc553855f31d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:15.792138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469c2c2-034c-454a-b0a0-bc553855f31d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:15.793620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469c2c2-034c-454a-b0a0-bc553855f31d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:15.811091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 317/644] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:15.812606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469c2c2-034c-454a-b0a0-bc553855f31d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:15.879097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43930ba5-6ad3-4c57-bd7a-ff8c1a0de31b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:15.917268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281f81e-b52f-4cc7-a126-b3d1c6ab47d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:15.918037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281f81e-b52f-4cc7-a126-b3d1c6ab47d1 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.918437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281f81e-b52f-4cc7-a126-b3d1c6ab47d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.918831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281f81e-b52f-4cc7-a126-b3d1c6ab47d1 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.919702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281f81e-b52f-4cc7-a126-b3d1c6ab47d1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:15.920120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281f81e-b52f-4cc7-a126-b3d1c6ab47d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:15.921659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281f81e-b52f-4cc7-a126-b3d1c6ab47d1 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:15.937647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469c2c2-034c-454a-b0a0-bc553855f31d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:15.944326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 328/645] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 3 14:39:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:15.944326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281f81e-b52f-4cc7-a126-b3d1c6ab47d1 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:15.990220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f548-707d-4e9d-b74b-653366d7749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:15.991401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f548-707d-4e9d-b74b-653366d7749d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:15.992350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f548-707d-4e9d-b74b-653366d7749d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:15.992901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f548-707d-4e9d-b74b-653366d7749d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:15.993825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:16.007278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:16.011845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535f548-707d-4e9d-b74b-653366d7749d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:16.014273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535f548-707d-4e9d-b74b-653366d7749d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-d535f548-707d-4e9d-b74b-653366d7749d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:16.032181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:16.033317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:16.034591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:16.034591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:16.034591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:16.037134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 318/646] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:16.038476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f548-707d-4e9d-b74b-653366d7749d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:16.061576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281f81e-b52f-4cc7-a126-b3d1c6ab47d1 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:16.098883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8e471-292e-4e30-8ee6-bc65e5fbba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:16.100347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8e471-292e-4e30-8ee6-bc65e5fbba41 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.100772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8e471-292e-4e30-8ee6-bc65e5fbba41 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.101079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8e471-292e-4e30-8ee6-bc65e5fbba41 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.108594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8e471-292e-4e30-8ee6-bc65e5fbba41 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:16.110074 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8e471-292e-4e30-8ee6-bc65e5fbba41 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-6bf8e471-292e-4e30-8ee6-bc65e5fbba41 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:16.122147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:16.123073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 329/647] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:16.124008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8e471-292e-4e30-8ee6-bc65e5fbba41 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:16.162706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f548-707d-4e9d-b74b-653366d7749d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:16.200691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7b082-3248-4ea1-8170-07e778df8eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:16.201292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7b082-3248-4ea1-8170-07e778df8eeb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.201591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7b082-3248-4ea1-8170-07e778df8eeb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.201860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7b082-3248-4ea1-8170-07e778df8eeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.230915 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8e471-292e-4e30-8ee6-bc65e5fbba41 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:16.233653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:16.261600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:16.269338 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6e24c-bc80-4ad4-892d-fa9aa489d92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:16.270002 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6e24c-bc80-4ad4-892d-fa9aa489d92f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.270281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6e24c-bc80-4ad4-892d-fa9aa489d92f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.270553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6e24c-bc80-4ad4-892d-fa9aa489d92f None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.271201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6e24c-bc80-4ad4-892d-fa9aa489d92f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:16.271502 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6e24c-bc80-4ad4-892d-fa9aa489d92f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:16.271777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7b082-3248-4ea1-8170-07e778df8eeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:16.272913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6e24c-bc80-4ad4-892d-fa9aa489d92f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:16.273248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7b082-3248-4ea1-8170-07e778df8eeb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:16.288256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 330/648] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:16.292466 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6e24c-bc80-4ad4-892d-fa9aa489d92f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:16.391398 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 319/649] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:39:16.396423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7b082-3248-4ea1-8170-07e778df8eeb None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:16.403561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6e24c-bc80-4ad4-892d-fa9aa489d92f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:16.442125 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105e4ae-9f73-4150-9b6f-4254ebc316df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:16.443100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105e4ae-9f73-4150-9b6f-4254ebc316df None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.443433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105e4ae-9f73-4150-9b6f-4254ebc316df None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.444016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105e4ae-9f73-4150-9b6f-4254ebc316df None admin] PATH_INFO: `/v3/roles/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.453955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105e4ae-9f73-4150-9b6f-4254ebc316df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-7105e4ae-9f73-4150-9b6f-4254ebc316df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:16.458644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:16.459583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:16.460460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 331/650] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:16.461155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105e4ae-9f73-4150-9b6f-4254ebc316df None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:16.485346 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:16.494612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:16.503507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7b082-3248-4ea1-8170-07e778df8eeb None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:16.536651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac02ea2-d924-443b-a02b-04863d2263b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:16.537197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac02ea2-d924-443b-a02b-04863d2263b7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.537479 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac02ea2-d924-443b-a02b-04863d2263b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.537782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac02ea2-d924-443b-a02b-04863d2263b7 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.539019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac02ea2-d924-443b-a02b-04863d2263b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:16.539327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac02ea2-d924-443b-a02b-04863d2263b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:16.540749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac02ea2-d924-443b-a02b-04863d2263b7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:16.557383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 320/651] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 3 14:39:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:16.558211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac02ea2-d924-443b-a02b-04863d2263b7 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:16.558948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105e4ae-9f73-4150-9b6f-4254ebc316df None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:16.591796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290cb33-7da7-40f8-ba1e-ab58c0c883de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:16.592413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290cb33-7da7-40f8-ba1e-ab58c0c883de None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.592691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290cb33-7da7-40f8-ba1e-ab58c0c883de None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.592960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290cb33-7da7-40f8-ba1e-ab58c0c883de None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.599518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290cb33-7da7-40f8-ba1e-ab58c0c883de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:16.601520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290cb33-7da7-40f8-ba1e-ab58c0c883de None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-2290cb33-7da7-40f8-ba1e-ab58c0c883de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:16.612267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:16.613109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:16.614045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:16.614045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:16.614045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:16.614045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:16.614045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:16.614045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:16.614045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:16.614045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:16.614045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 332/652] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:16.615264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d6432-d7ae-4555-982e-2cd132482af1 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.615264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d6432-d7ae-4555-982e-2cd132482af1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.615264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d6432-d7ae-4555-982e-2cd132482af1 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.616607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 333/653] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:16.617140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d6432-d7ae-4555-982e-2cd132482af1 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:16.659526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac02ea2-d924-443b-a02b-04863d2263b7 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:16.692790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b4452-6ccd-432b-852f-5be284199c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:16.693423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b4452-6ccd-432b-852f-5be284199c45 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.693735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b4452-6ccd-432b-852f-5be284199c45 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.694003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b4452-6ccd-432b-852f-5be284199c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.701310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:16.703884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b4452-6ccd-432b-852f-5be284199c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:16.705357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b4452-6ccd-432b-852f-5be284199c45 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:16.717658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d6432-d7ae-4555-982e-2cd132482af1 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-d87b4452-6ccd-432b-852f-5be284199c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:16.720753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:16.721719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:16.722653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:16.722653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:16.722653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:16.722653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 321/654] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:16.722653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b4452-6ccd-432b-852f-5be284199c45 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:16.725219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:16.753634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef384de1-e42b-4217-9f81-f877e5ff2b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:16.754243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef384de1-e42b-4217-9f81-f877e5ff2b18 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.754507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef384de1-e42b-4217-9f81-f877e5ff2b18 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.754775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef384de1-e42b-4217-9f81-f877e5ff2b18 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.755434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef384de1-e42b-4217-9f81-f877e5ff2b18 None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:16.755730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef384de1-e42b-4217-9f81-f877e5ff2b18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:16.757081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef384de1-e42b-4217-9f81-f877e5ff2b18 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:16.764680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 334/655] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:16.765443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef384de1-e42b-4217-9f81-f877e5ff2b18 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:16.822224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b4452-6ccd-432b-852f-5be284199c45 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:16.854912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b534dd-f6f2-481a-9209-04440df7f13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:16.855511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b534dd-f6f2-481a-9209-04440df7f13b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.855795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b534dd-f6f2-481a-9209-04440df7f13b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.856068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b534dd-f6f2-481a-9209-04440df7f13b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.865108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef384de1-e42b-4217-9f81-f877e5ff2b18 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:16.899537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3649c4-c84f-4765-bfde-94fb19fcef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:16.900128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3649c4-c84f-4765-bfde-94fb19fcef99 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.900415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3649c4-c84f-4765-bfde-94fb19fcef99 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.900684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3649c4-c84f-4765-bfde-94fb19fcef99 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.901357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3649c4-c84f-4765-bfde-94fb19fcef99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:16.901691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3649c4-c84f-4765-bfde-94fb19fcef99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:16.903103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3649c4-c84f-4765-bfde-94fb19fcef99 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:16.915587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 335/656] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:16.916736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae555af3-3250-4cfe-af54-1fac267fdbd9 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:16.917103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae555af3-3250-4cfe-af54-1fac267fdbd9 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:16.917298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae555af3-3250-4cfe-af54-1fac267fdbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:16.922972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b534dd-f6f2-481a-9209-04440df7f13b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:16.924399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b534dd-f6f2-481a-9209-04440df7f13b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:16.926203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:16.929547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:16.934677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ae555af3-3250-4cfe-af54-1fac267fdbd9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:16.935864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ae555af3-3250-4cfe-af54-1fac267fdbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:17.000377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 336/657] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:17.001199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae555af3-3250-4cfe-af54-1fac267fdbd9 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.025973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 322/658] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:39:17.031028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b534dd-f6f2-481a-9209-04440df7f13b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.101158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae555af3-3250-4cfe-af54-1fac267fdbd9 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.129559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b534dd-f6f2-481a-9209-04440df7f13b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.131026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:17.137127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e13a9-7d90-4fbc-98dc-ec4762ae1beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.137757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e13a9-7d90-4fbc-98dc-ec4762ae1beb None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.138036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e13a9-7d90-4fbc-98dc-ec4762ae1beb None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.138306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e13a9-7d90-4fbc-98dc-ec4762ae1beb None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.138945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e13a9-7d90-4fbc-98dc-ec4762ae1beb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:17.139243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e13a9-7d90-4fbc-98dc-ec4762ae1beb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:17.140611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e13a9-7d90-4fbc-98dc-ec4762ae1beb None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:17.143190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:17.158203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 337/659] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:17.158203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e13a9-7d90-4fbc-98dc-ec4762ae1beb None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.170813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4290bd7-f3c8-4466-ae30-7998c33c203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.172087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4290bd7-f3c8-4466-ae30-7998c33c203e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.172509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4290bd7-f3c8-4466-ae30-7998c33c203e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.172787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4290bd7-f3c8-4466-ae30-7998c33c203e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.183241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4290bd7-f3c8-4466-ae30-7998c33c203e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:17.184665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4290bd7-f3c8-4466-ae30-7998c33c203e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-f4290bd7-f3c8-4466-ae30-7998c33c203e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:17.199023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:17.200153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:17.201026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:17.201026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:17.201026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:17.201026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 323/660] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:17.201026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4290bd7-f3c8-4466-ae30-7998c33c203e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.264274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e13a9-7d90-4fbc-98dc-ec4762ae1beb None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.302944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf988f-57d7-42a1-9332-17b02b202741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.303536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf988f-57d7-42a1-9332-17b02b202741 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.303814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf988f-57d7-42a1-9332-17b02b202741 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.304632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf988f-57d7-42a1-9332-17b02b202741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.305362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf988f-57d7-42a1-9332-17b02b202741 None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=1a4a95ae5ee84f259ffcd7528b0fa505) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:17.305464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4290bd7-f3c8-4466-ae30-7998c33c203e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.305700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf988f-57d7-42a1-9332-17b02b202741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:17.307139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf988f-57d7-42a1-9332-17b02b202741 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:17.312691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 338/661] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/role_assignments?user.id=1a4a95ae5ee84f259ffcd7528b0fa505&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:17.313465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf988f-57d7-42a1-9332-17b02b202741 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.343291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29df66-6a74-4e6a-9880-85973e44dabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.343948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29df66-6a74-4e6a-9880-85973e44dabe None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.344245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29df66-6a74-4e6a-9880-85973e44dabe None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.344505 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29df66-6a74-4e6a-9880-85973e44dabe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.358679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29df66-6a74-4e6a-9880-85973e44dabe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:17.360087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29df66-6a74-4e6a-9880-85973e44dabe None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-ab29df66-6a74-4e6a-9880-85973e44dabe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:17.377041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:17.378206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:17.379106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:17.379106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:17.379106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:17.379106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 324/662] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:17.379106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29df66-6a74-4e6a-9880-85973e44dabe None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.388555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:17.392416 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:17.436178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf988f-57d7-42a1-9332-17b02b202741 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.473738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d031a1-1263-45e0-ab32-81ac1468cb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.474676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d031a1-1263-45e0-ab32-81ac1468cb7d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.475131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d031a1-1263-45e0-ab32-81ac1468cb7d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.475548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d031a1-1263-45e0-ab32-81ac1468cb7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.482981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d031a1-1263-45e0-ab32-81ac1468cb7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:17.484722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d031a1-1263-45e0-ab32-81ac1468cb7d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:17.494402 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29df66-6a74-4e6a-9880-85973e44dabe None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-58d031a1-1263-45e0-ab32-81ac1468cb7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:17.500859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:17.501823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:17.502776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:17.502776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:17.502776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:17.502776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:17.502776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:17.502776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:17.502776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:17.502776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:17.502776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 339/663] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:17.502776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d031a1-1263-45e0-ab32-81ac1468cb7d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.543045 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c5f39-f69b-40aa-bee7-64589a211ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.543656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c5f39-f69b-40aa-bee7-64589a211ad8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.543934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c5f39-f69b-40aa-bee7-64589a211ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.544202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c5f39-f69b-40aa-bee7-64589a211ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.544847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c5f39-f69b-40aa-bee7-64589a211ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:17.545669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c5f39-f69b-40aa-bee7-64589a211ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:17.547419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c5f39-f69b-40aa-bee7-64589a211ad8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:17.562447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 325/664] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:17.563901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c5f39-f69b-40aa-bee7-64589a211ad8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.615257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d031a1-1263-45e0-ab32-81ac1468cb7d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.632634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:17.637030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:17.652212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd051f-6192-498a-b77b-2c594d0189f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.652846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd051f-6192-498a-b77b-2c594d0189f5 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.653113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd051f-6192-498a-b77b-2c594d0189f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.653388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd051f-6192-498a-b77b-2c594d0189f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.663732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd051f-6192-498a-b77b-2c594d0189f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-9fdd051f-6192-498a-b77b-2c594d0189f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:17.669122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:17.670263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 340/665] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:17.671092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd051f-6192-498a-b77b-2c594d0189f5 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.677642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c5f39-f69b-40aa-bee7-64589a211ad8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.714439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1d5d4-b589-4580-8fbd-fadbf9b31a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.715056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1d5d4-b589-4580-8fbd-fadbf9b31a82 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.715340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1d5d4-b589-4580-8fbd-fadbf9b31a82 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.715607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1d5d4-b589-4580-8fbd-fadbf9b31a82 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.716254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1d5d4-b589-4580-8fbd-fadbf9b31a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:17.716557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1d5d4-b589-4580-8fbd-fadbf9b31a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:17.717956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1d5d4-b589-4580-8fbd-fadbf9b31a82 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:17.729862 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 326/666] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:17.730995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1d5d4-b589-4580-8fbd-fadbf9b31a82 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.782323 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd051f-6192-498a-b77b-2c594d0189f5 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.818234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dfdb4-2473-4dce-8d02-a657e5802309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.819287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dfdb4-2473-4dce-8d02-a657e5802309 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.819774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dfdb4-2473-4dce-8d02-a657e5802309 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.820223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dfdb4-2473-4dce-8d02-a657e5802309 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.821110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dfdb4-2473-4dce-8d02-a657e5802309 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:17.821656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dfdb4-2473-4dce-8d02-a657e5802309 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:17.823336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dfdb4-2473-4dce-8d02-a657e5802309 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:17.833235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1d5d4-b589-4580-8fbd-fadbf9b31a82 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.836432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 341/667] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:17.837286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dfdb4-2473-4dce-8d02-a657e5802309 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.873949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:17.881536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:17.883567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9860f5-6007-428e-8842-8cede07fa74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.884180 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9860f5-6007-428e-8842-8cede07fa74a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.884466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9860f5-6007-428e-8842-8cede07fa74a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.884737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9860f5-6007-428e-8842-8cede07fa74a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.892736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9860f5-6007-428e-8842-8cede07fa74a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:17.894202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9860f5-6007-428e-8842-8cede07fa74a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-4e9860f5-6007-428e-8842-8cede07fa74a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:17.906102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:17.907024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:17.907935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:17.907935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:17.907935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:17.907935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:17.907935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:17.907935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:17.907935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:17.907935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:17.907935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 14:39:17.907935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:17.910392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 327/668] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:17.911369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9860f5-6007-428e-8842-8cede07fa74a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:17.948386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dfdb4-2473-4dce-8d02-a657e5802309 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:17.984910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258f4ec-fa2f-456c-b586-3254c2934d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:17.985647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258f4ec-fa2f-456c-b586-3254c2934d94 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:17.985930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258f4ec-fa2f-456c-b586-3254c2934d94 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:17.986202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258f4ec-fa2f-456c-b586-3254c2934d94 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:17.986866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258f4ec-fa2f-456c-b586-3254c2934d94 None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:17.987166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258f4ec-fa2f-456c-b586-3254c2934d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:17.988532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258f4ec-fa2f-456c-b586-3254c2934d94 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:18.002713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 342/669] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:18.002713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258f4ec-fa2f-456c-b586-3254c2934d94 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:18.022529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9860f5-6007-428e-8842-8cede07fa74a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:18.059925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dd9f5-7813-4644-90e0-ebe0a7a5c7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:18.060555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dd9f5-7813-4644-90e0-ebe0a7a5c7cb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:18.060837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dd9f5-7813-4644-90e0-ebe0a7a5c7cb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:18.061110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dd9f5-7813-4644-90e0-ebe0a7a5c7cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:18.069206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46dd9f5-7813-4644-90e0-ebe0a7a5c7cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:18.071563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46dd9f5-7813-4644-90e0-ebe0a7a5c7cb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-f46dd9f5-7813-4644-90e0-ebe0a7a5c7cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:18.084064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:18.084963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:18.086399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:18.086399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:18.086399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:18.086399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:18.086399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:18.086399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:18.086399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:18.086399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:18.091349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 328/670] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:18.092068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dd9f5-7813-4644-90e0-ebe0a7a5c7cb None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:18.103433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258f4ec-fa2f-456c-b586-3254c2934d94 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:18.104976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:18.139685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ecbb-8ab3-435f-b49e-96ea90220546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:18.140214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ecbb-8ab3-435f-b49e-96ea90220546 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:18.140483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ecbb-8ab3-435f-b49e-96ea90220546 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:18.140757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ecbb-8ab3-435f-b49e-96ea90220546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:18.141467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8ecbb-8ab3-435f-b49e-96ea90220546 None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=64a9421bec3b4b9e89dec35d7556d025) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:18.141789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8ecbb-8ab3-435f-b49e-96ea90220546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:18.143206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8ecbb-8ab3-435f-b49e-96ea90220546 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:18.144784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:18.149770 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 343/671] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:17 2022] GET /identity/v3/role_assignments?user.id=64a9421bec3b4b9e89dec35d7556d025&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:18.150538 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ecbb-8ab3-435f-b49e-96ea90220546 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:18.233424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dd9f5-7813-4644-90e0-ebe0a7a5c7cb None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:18.265681 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ecbb-8ab3-435f-b49e-96ea90220546 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:18.281549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fbdcd-6fc7-487a-9bd3-95cb94fbd3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:18.282185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fbdcd-6fc7-487a-9bd3-95cb94fbd3f6 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:18.282460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fbdcd-6fc7-487a-9bd3-95cb94fbd3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:18.282733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fbdcd-6fc7-487a-9bd3-95cb94fbd3f6 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:18.283479 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fbdcd-6fc7-487a-9bd3-95cb94fbd3f6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:18.283778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fbdcd-6fc7-487a-9bd3-95cb94fbd3f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:18.285244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fbdcd-6fc7-487a-9bd3-95cb94fbd3f6 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:18.317682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 329/672] 10.222.0.46 () {56 vars in 1139 bytes} [Mon Oct 3 14:39:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:18.318634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fbdcd-6fc7-487a-9bd3-95cb94fbd3f6 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:18.322049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9568d1f-7b7d-4b76-8b27-a3caa68224e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:18.323087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9568d1f-7b7d-4b76-8b27-a3caa68224e6 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:18.323368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9568d1f-7b7d-4b76-8b27-a3caa68224e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:18.324662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9568d1f-7b7d-4b76-8b27-a3caa68224e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:18.340064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9568d1f-7b7d-4b76-8b27-a3caa68224e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:18.342109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9568d1f-7b7d-4b76-8b27-a3caa68224e6 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-e9568d1f-7b7d-4b76-8b27-a3caa68224e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:18.367965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:18.368973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:18.369957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:18.369957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:18.369957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:18.369957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 344/673] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:18.370191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9568d1f-7b7d-4b76-8b27-a3caa68224e6 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:18.408330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:18.488395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fbdcd-6fc7-487a-9bd3-95cb94fbd3f6 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:18.488395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:18.541753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9568d1f-7b7d-4b76-8b27-a3caa68224e6 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:18.567448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c46ad-c801-428c-99dd-c21e5533fefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:18.568524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c46ad-c801-428c-99dd-c21e5533fefd None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:18.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c46ad-c801-428c-99dd-c21e5533fefd None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:18.571304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c46ad-c801-428c-99dd-c21e5533fefd None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:18.572033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c46ad-c801-428c-99dd-c21e5533fefd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:18.572348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c46ad-c801-428c-99dd-c21e5533fefd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:18.575296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c46ad-c801-428c-99dd-c21e5533fefd None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:18.587160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65da19-fe69-46b1-af1f-371f5f3205e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:18.587760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65da19-fe69-46b1-af1f-371f5f3205e9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:18.588035 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65da19-fe69-46b1-af1f-371f5f3205e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:18.588303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65da19-fe69-46b1-af1f-371f5f3205e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:18.616759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 330/674] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:18.617388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c46ad-c801-428c-99dd-c21e5533fefd None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:18.684605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65da19-fe69-46b1-af1f-371f5f3205e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:18.686041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65da19-fe69-46b1-af1f-371f5f3205e9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:18.723993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:18.758510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c46ad-c801-428c-99dd-c21e5533fefd None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:18.830248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed032-2a65-42a2-b931-cd01823a02db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:18.830867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed032-2a65-42a2-b931-cd01823a02db None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:18.831142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed032-2a65-42a2-b931-cd01823a02db None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:18.831411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed032-2a65-42a2-b931-cd01823a02db None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:18.832142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eed032-2a65-42a2-b931-cd01823a02db None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:18.832446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eed032-2a65-42a2-b931-cd01823a02db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:18.833844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eed032-2a65-42a2-b931-cd01823a02db None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:18.860549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 331/675] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:18.860549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed032-2a65-42a2-b931-cd01823a02db None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:18.903320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 345/676] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:39:18.910164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65da19-fe69-46b1-af1f-371f5f3205e9 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:18.915248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:19.014898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed032-2a65-42a2-b931-cd01823a02db None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:19.040131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65da19-fe69-46b1-af1f-371f5f3205e9 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:19.059905 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52953bd4-19fe-44f5-9db2-0ea583a0a0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:19.061031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52953bd4-19fe-44f5-9db2-0ea583a0a0a8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:19.061590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52953bd4-19fe-44f5-9db2-0ea583a0a0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:19.062223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52953bd4-19fe-44f5-9db2-0ea583a0a0a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:19.063271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:19.064893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52953bd4-19fe-44f5-9db2-0ea583a0a0a8 None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=dd04df68f9d84fc4bb981c8b69cc771a) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:19.065460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52953bd4-19fe-44f5-9db2-0ea583a0a0a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:19.067982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52953bd4-19fe-44f5-9db2-0ea583a0a0a8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:19.077568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 332/677] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:18 2022] GET /identity/v3/role_assignments?user.id=dd04df68f9d84fc4bb981c8b69cc771a&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:19.079298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52953bd4-19fe-44f5-9db2-0ea583a0a0a8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:19.084787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af0aa5-afad-4416-92d6-5261eea1d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:19.085674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af0aa5-afad-4416-92d6-5261eea1d8ed None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:19.086053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af0aa5-afad-4416-92d6-5261eea1d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:19.086425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af0aa5-afad-4416-92d6-5261eea1d8ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:19.096545 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af0aa5-afad-4416-92d6-5261eea1d8ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:19.098539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af0aa5-afad-4416-92d6-5261eea1d8ed None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-a6af0aa5-afad-4416-92d6-5261eea1d8ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:19.117381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:19.121754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:19.123148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 346/678] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:19.124874 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af0aa5-afad-4416-92d6-5261eea1d8ed None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:19.261058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:19.276602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52953bd4-19fe-44f5-9db2-0ea583a0a0a8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:19.314855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70e09e-d344-4422-99d1-d83051aade08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:19.315585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70e09e-d344-4422-99d1-d83051aade08 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:19.315936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70e09e-d344-4422-99d1-d83051aade08 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:19.316278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70e09e-d344-4422-99d1-d83051aade08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:19.330605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af0aa5-afad-4416-92d6-5261eea1d8ed None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:19.331669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70e09e-d344-4422-99d1-d83051aade08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:19.333245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70e09e-d344-4422-99d1-d83051aade08 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-1d70e09e-d344-4422-99d1-d83051aade08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:19.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:19.351337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:19.352217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:19.352217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:19.352217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:19.352217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 333/679] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:19.357390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70e09e-d344-4422-99d1-d83051aade08 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:19.385532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac1c85-ec16-47b2-9429-83b7360da46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:19.385532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac1c85-ec16-47b2-9429-83b7360da46e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:19.385532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac1c85-ec16-47b2-9429-83b7360da46e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:19.385532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac1c85-ec16-47b2-9429-83b7360da46e None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:19.385973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac1c85-ec16-47b2-9429-83b7360da46e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:19.385973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac1c85-ec16-47b2-9429-83b7360da46e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:19.385973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac1c85-ec16-47b2-9429-83b7360da46e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:19.402523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 347/680] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 3 14:39:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:19.408392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac1c85-ec16-47b2-9429-83b7360da46e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:19.448309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:19.497408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70e09e-d344-4422-99d1-d83051aade08 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:19.535518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a23391-e5d1-4aa7-8d7c-de56bfa49354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:19.536105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a23391-e5d1-4aa7-8d7c-de56bfa49354 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:19.536385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a23391-e5d1-4aa7-8d7c-de56bfa49354 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:19.536708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a23391-e5d1-4aa7-8d7c-de56bfa49354 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:19.537366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a23391-e5d1-4aa7-8d7c-de56bfa49354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:19.537702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a23391-e5d1-4aa7-8d7c-de56bfa49354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:19.539123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a23391-e5d1-4aa7-8d7c-de56bfa49354 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:19.539914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac1c85-ec16-47b2-9429-83b7360da46e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:19.548196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:19.554988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 334/681] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:19.559531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a23391-e5d1-4aa7-8d7c-de56bfa49354 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:19.587171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c5ab9-1deb-425d-b2a5-30c77da5884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:19.587777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c5ab9-1deb-425d-b2a5-30c77da5884b None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:19.588107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c5ab9-1deb-425d-b2a5-30c77da5884b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:19.588373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c5ab9-1deb-425d-b2a5-30c77da5884b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:19.682543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c5ab9-1deb-425d-b2a5-30c77da5884b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:19.684288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c5ab9-1deb-425d-b2a5-30c77da5884b None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:19.692247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a23391-e5d1-4aa7-8d7c-de56bfa49354 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:19.728702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b622a-bb74-477d-be71-515d4d4f4357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:19.729533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:19.729958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b622a-bb74-477d-be71-515d4d4f4357 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:19.730605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b622a-bb74-477d-be71-515d4d4f4357 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:19.731159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b622a-bb74-477d-be71-515d4d4f4357 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:19.738410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b622a-bb74-477d-be71-515d4d4f4357 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:19.740405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b622a-bb74-477d-be71-515d4d4f4357 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-cb9b622a-bb74-477d-be71-515d4d4f4357 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:19.752218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:19.753085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:19.753992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:19.753992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:19.753992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:19.753992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:19.753992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:19.753992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:19.753992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:19.753992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:19.760338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 335/682] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:19.761847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d852b03-ffa6-4a2b-a72b-b9532b0b6c53 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:19.762274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d852b03-ffa6-4a2b-a72b-b9532b0b6c53 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:19.762706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d852b03-ffa6-4a2b-a72b-b9532b0b6c53 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:19.764577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 336/683] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:19.765525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d852b03-ffa6-4a2b-a72b-b9532b0b6c53 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:19.814407 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 348/684] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 14:39:19.818340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f036c4-af2f-46f2-9cf3-1fbc4f4020e2 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:19.818769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f036c4-af2f-46f2-9cf3-1fbc4f4020e2 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:19.819281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f036c4-af2f-46f2-9cf3-1fbc4f4020e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:19.839805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-15f036c4-af2f-46f2-9cf3-1fbc4f4020e2 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:19.840036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:19.841859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-15f036c4-af2f-46f2-9cf3-1fbc4f4020e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:19.886591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d852b03-ffa6-4a2b-a72b-b9532b0b6c53 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:19.920921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 349/685] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:19.921712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b8ea2-6616-464a-b71b-f7162cfcc849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:19.921931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f036c4-af2f-46f2-9cf3-1fbc4f4020e2 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:19.922718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b8ea2-6616-464a-b71b-f7162cfcc849 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:19.923190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b8ea2-6616-464a-b71b-f7162cfcc849 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:19.923665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b8ea2-6616-464a-b71b-f7162cfcc849 None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:19.924546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b8ea2-6616-464a-b71b-f7162cfcc849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:19.925041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b8ea2-6616-464a-b71b-f7162cfcc849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:19.926766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b8ea2-6616-464a-b71b-f7162cfcc849 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:19.945647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 337/686] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:19.947415 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b8ea2-6616-464a-b71b-f7162cfcc849 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:19.994180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:20.032677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f036c4-af2f-46f2-9cf3-1fbc4f4020e2 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:20.050507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b8ea2-6616-464a-b71b-f7162cfcc849 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:20.070931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e996e-1ad1-45b9-87bd-fbf2913532c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:20.071490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e996e-1ad1-45b9-87bd-fbf2913532c3 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:20.071769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e996e-1ad1-45b9-87bd-fbf2913532c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:20.072040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e996e-1ad1-45b9-87bd-fbf2913532c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:20.073740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:20.085603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e996e-1ad1-45b9-87bd-fbf2913532c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:20.087240 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e996e-1ad1-45b9-87bd-fbf2913532c3 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:20.088914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f72a7-f58e-405a-8f3e-58978fad35de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:20.089647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f72a7-f58e-405a-8f3e-58978fad35de None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:20.090019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f72a7-f58e-405a-8f3e-58978fad35de None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:20.090383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f72a7-f58e-405a-8f3e-58978fad35de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:20.091254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f72a7-f58e-405a-8f3e-58978fad35de None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=05e9009410a54ce3ab88fa0cac87a7f9) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:20.091677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f72a7-f58e-405a-8f3e-58978fad35de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:20.093192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f72a7-f58e-405a-8f3e-58978fad35de None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:20.104920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 338/687] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:19 2022] GET /identity/v3/role_assignments?user.id=05e9009410a54ce3ab88fa0cac87a7f9&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-9e5e996e-1ad1-45b9-87bd-fbf2913532c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:20.105130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:20.106245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:20.107288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:20.107288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:20.107288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:20.108952 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 350/688] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:20.115042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f72a7-f58e-405a-8f3e-58978fad35de None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:20.268510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f72a7-f58e-405a-8f3e-58978fad35de None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:20.317935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808042-ae3e-46dc-9f22-327de7b95a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:20.318550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808042-ae3e-46dc-9f22-327de7b95a2c None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:20.318836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808042-ae3e-46dc-9f22-327de7b95a2c None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:20.319110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808042-ae3e-46dc-9f22-327de7b95a2c None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:20.319819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99808042-ae3e-46dc-9f22-327de7b95a2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:20.320126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99808042-ae3e-46dc-9f22-327de7b95a2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:20.321836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99808042-ae3e-46dc-9f22-327de7b95a2c None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:20.323015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e996e-1ad1-45b9-87bd-fbf2913532c3 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:20.341754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 339/689] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:20.352117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808042-ae3e-46dc-9f22-327de7b95a2c None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:20.400870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:20.456929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e996e-1ad1-45b9-87bd-fbf2913532c3 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:20.468302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808042-ae3e-46dc-9f22-327de7b95a2c None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:20.479086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:20.504997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92189e34-370d-492d-8025-49ef02b06e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:20.506208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92189e34-370d-492d-8025-49ef02b06e2d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:20.506497 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92189e34-370d-492d-8025-49ef02b06e2d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:20.507121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92189e34-370d-492d-8025-49ef02b06e2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:20.511779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f044e6e-cf6a-4056-9bf3-165e5c464f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:20.512497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f044e6e-cf6a-4056-9bf3-165e5c464f47 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:20.512781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f044e6e-cf6a-4056-9bf3-165e5c464f47 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:20.513050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f044e6e-cf6a-4056-9bf3-165e5c464f47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:20.521215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f044e6e-cf6a-4056-9bf3-165e5c464f47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:20.523409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f044e6e-cf6a-4056-9bf3-165e5c464f47 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:20.523685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92189e34-370d-492d-8025-49ef02b06e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-92189e34-370d-492d-8025-49ef02b06e2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:20.530587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:20.531521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:20.532351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 351/690] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-3f044e6e-cf6a-4056-9bf3-165e5c464f47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:20.539247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:20.540119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:20.541069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:20.541069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:20.541069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:20.541069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:20.541069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:20.541069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:20.541069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:20.541069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:20.541069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 340/691] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:20.543598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92189e34-370d-492d-8025-49ef02b06e2d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:20.547488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f044e6e-cf6a-4056-9bf3-165e5c464f47 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:20.712124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f044e6e-cf6a-4056-9bf3-165e5c464f47 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:20.715445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:20.733240 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92189e34-370d-492d-8025-49ef02b06e2d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:20.759694 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a63e3-be9e-431e-ad26-a41a40a2c6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:20.761167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a63e3-be9e-431e-ad26-a41a40a2c6cb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:20.761814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a63e3-be9e-431e-ad26-a41a40a2c6cb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:20.762333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a63e3-be9e-431e-ad26-a41a40a2c6cb None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:20.763428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a63e3-be9e-431e-ad26-a41a40a2c6cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:20.764284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a63e3-be9e-431e-ad26-a41a40a2c6cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:20.766234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a63e3-be9e-431e-ad26-a41a40a2c6cb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:20.768584 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e142c-a9c6-4df9-9720-53ac6e77476d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:20.769342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e142c-a9c6-4df9-9720-53ac6e77476d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:20.769460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e142c-a9c6-4df9-9720-53ac6e77476d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:20.769742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e142c-a9c6-4df9-9720-53ac6e77476d None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:20.770404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e142c-a9c6-4df9-9720-53ac6e77476d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:20.770707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e142c-a9c6-4df9-9720-53ac6e77476d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:20.772079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e142c-a9c6-4df9-9720-53ac6e77476d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:20.781387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 352/692] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:20.782344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 341/693] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:20.782612 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0aaf6-5da0-4b2d-b2ab-335caf87cab3 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:20.782894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0aaf6-5da0-4b2d-b2ab-335caf87cab3 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:20.783165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0aaf6-5da0-4b2d-b2ab-335caf87cab3 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:20.784462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 353/694] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:20.789755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a63e3-be9e-431e-ad26-a41a40a2c6cb None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:20.790124 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7580d8-fd4f-4625-9fac-0d6514009291 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:20.790411 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7580d8-fd4f-4625-9fac-0d6514009291 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:20.790677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7580d8-fd4f-4625-9fac-0d6514009291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:20.808831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5f7580d8-fd4f-4625-9fac-0d6514009291 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:20.810080 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5f7580d8-fd4f-4625-9fac-0d6514009291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:20.848194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:20.889524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 354/695] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:39:20.889524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7580d8-fd4f-4625-9fac-0d6514009291 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:20.926794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a63e3-be9e-431e-ad26-a41a40a2c6cb None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:20.990210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67378be-f0f8-439e-a8d7-8523ed03aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:20.991888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67378be-f0f8-439e-a8d7-8523ed03aac7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:20.992350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67378be-f0f8-439e-a8d7-8523ed03aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:20.993208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67378be-f0f8-439e-a8d7-8523ed03aac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:20.995147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67378be-f0f8-439e-a8d7-8523ed03aac7 None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=6fbf23ae20ee4f6c979e256d2d4daa5f) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:20.996930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67378be-f0f8-439e-a8d7-8523ed03aac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:20.998995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67378be-f0f8-439e-a8d7-8523ed03aac7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:21.012450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 342/696] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:20 2022] GET /identity/v3/role_assignments?user.id=6fbf23ae20ee4f6c979e256d2d4daa5f&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:21.052399 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7580d8-fd4f-4625-9fac-0d6514009291 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:21.102483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c761fb-0fe0-4adc-ae97-ce01d37f24c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:21.103381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c761fb-0fe0-4adc-ae97-ce01d37f24c4 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:21.103807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c761fb-0fe0-4adc-ae97-ce01d37f24c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:21.104246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c761fb-0fe0-4adc-ae97-ce01d37f24c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:21.123275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c761fb-0fe0-4adc-ae97-ce01d37f24c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:21.124977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c761fb-0fe0-4adc-ae97-ce01d37f24c4 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-a4c761fb-0fe0-4adc-ae97-ce01d37f24c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:21.143352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:21.144525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:21.149692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:21.149692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:21.149692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:21.151286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 355/697] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:21.156830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67378be-f0f8-439e-a8d7-8523ed03aac7 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:21.191983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:21.292450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67378be-f0f8-439e-a8d7-8523ed03aac7 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:21.292619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c761fb-0fe0-4adc-ae97-ce01d37f24c4 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:21.339814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:21.340878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd4029-27cf-4d8d-b24f-2cbd978366fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:21.342137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd4029-27cf-4d8d-b24f-2cbd978366fa None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:21.342868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd4029-27cf-4d8d-b24f-2cbd978366fa None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:21.343333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd4029-27cf-4d8d-b24f-2cbd978366fa None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:21.344198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd4029-27cf-4d8d-b24f-2cbd978366fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:21.344868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd4029-27cf-4d8d-b24f-2cbd978366fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:21.347033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd4029-27cf-4d8d-b24f-2cbd978366fa None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:21.363861 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 343/698] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:21.370670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd4029-27cf-4d8d-b24f-2cbd978366fa None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:21.442903 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c761fb-0fe0-4adc-ae97-ce01d37f24c4 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:21.484460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c4257-518b-4a89-888f-3ac4a8c8df2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:21.485105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c4257-518b-4a89-888f-3ac4a8c8df2e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:21.485346 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c4257-518b-4a89-888f-3ac4a8c8df2e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:21.485649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c4257-518b-4a89-888f-3ac4a8c8df2e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:21.486231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c4257-518b-4a89-888f-3ac4a8c8df2e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:21.489519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c4257-518b-4a89-888f-3ac4a8c8df2e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:21.496693 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-f54c4257-518b-4a89-888f-3ac4a8c8df2e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:21.507792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:21.508845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:21.509750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 356/699] 10.222.0.46 () {56 vars in 1129 bytes} [Mon Oct 3 14:39:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:21.514867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c4257-518b-4a89-888f-3ac4a8c8df2e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:21.528776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd4029-27cf-4d8d-b24f-2cbd978366fa None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:21.573369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88802050-52fc-415d-a537-ff4e3759e303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:21.574654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88802050-52fc-415d-a537-ff4e3759e303 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:21.575157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88802050-52fc-415d-a537-ff4e3759e303 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:21.575622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88802050-52fc-415d-a537-ff4e3759e303 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:21.585280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88802050-52fc-415d-a537-ff4e3759e303 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:21.587661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88802050-52fc-415d-a537-ff4e3759e303 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-88802050-52fc-415d-a537-ff4e3759e303 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:21.602255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:21.603202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:21.604117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:21.604117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:21.604117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:21.604117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:21.604117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:21.604117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:21.604117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:21.604117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:21.604117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:21.604117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:21.606900 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 344/700] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 3 14:39:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:21.612628 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88802050-52fc-415d-a537-ff4e3759e303 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:21.617294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:21.636062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c4257-518b-4a89-888f-3ac4a8c8df2e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:21.673568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ecb11-3ed6-4662-9f82-364ff1fd8862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:21.674326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ecb11-3ed6-4662-9f82-364ff1fd8862 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:21.674802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ecb11-3ed6-4662-9f82-364ff1fd8862 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:21.675166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ecb11-3ed6-4662-9f82-364ff1fd8862 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:21.675979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ecb11-3ed6-4662-9f82-364ff1fd8862 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:21.676466 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ecb11-3ed6-4662-9f82-364ff1fd8862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:21.678097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ecb11-3ed6-4662-9f82-364ff1fd8862 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:21.683854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 357/701] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 3 14:39:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:21.691739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ecb11-3ed6-4662-9f82-364ff1fd8862 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:21.729764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88802050-52fc-415d-a537-ff4e3759e303 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:21.767626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3f89c-6fee-40f9-a2fb-48151e5176ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:21.768515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3f89c-6fee-40f9-a2fb-48151e5176ca None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:21.768939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3f89c-6fee-40f9-a2fb-48151e5176ca None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:21.769354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3f89c-6fee-40f9-a2fb-48151e5176ca None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:21.770376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3f89c-6fee-40f9-a2fb-48151e5176ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:21.770843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3f89c-6fee-40f9-a2fb-48151e5176ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:21.772576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3f89c-6fee-40f9-a2fb-48151e5176ca None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:21.793070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 345/702] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 3 14:39:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:21.798702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3f89c-6fee-40f9-a2fb-48151e5176ca None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:21.808828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:21.913790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3f89c-6fee-40f9-a2fb-48151e5176ca None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:21.945932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ecb11-3ed6-4662-9f82-364ff1fd8862 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:21.952795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82918d43-0817-40a9-a058-a2c75545059d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:21.953948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82918d43-0817-40a9-a058-a2c75545059d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:21.954369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82918d43-0817-40a9-a058-a2c75545059d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:21.954782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82918d43-0817-40a9-a058-a2c75545059d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:21.990586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02f813-17fa-4160-a491-56e2fc1dfc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:21.992155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02f813-17fa-4160-a491-56e2fc1dfc9d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:21.992433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02f813-17fa-4160-a491-56e2fc1dfc9d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:21.992830 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02f813-17fa-4160-a491-56e2fc1dfc9d None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:21.993469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02f813-17fa-4160-a491-56e2fc1dfc9d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:21.993887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02f813-17fa-4160-a491-56e2fc1dfc9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:21.996544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02f813-17fa-4160-a491-56e2fc1dfc9d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:22.005677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 358/703] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 3 14:39:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:22.007143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1884b0-7772-4df2-aedb-04af5ce3e3f8 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:22.007421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1884b0-7772-4df2-aedb-04af5ce3e3f8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:22.007691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1884b0-7772-4df2-aedb-04af5ce3e3f8 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:22.009400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 359/704] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:22.024041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009345d-226f-4be2-9ebd-3a99278e27ad None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:22.024370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009345d-226f-4be2-9ebd-3a99278e27ad None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:22.024590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009345d-226f-4be2-9ebd-3a99278e27ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:22.050175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-1009345d-226f-4be2-9ebd-3a99278e27ad None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:22.051370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1009345d-226f-4be2-9ebd-3a99278e27ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:22.055819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82918d43-0817-40a9-a058-a2c75545059d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:22.057276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82918d43-0817-40a9-a058-a2c75545059d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:22.066296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:22.074240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:22.173490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 346/705] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 14:39:22.180115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82918d43-0817-40a9-a058-a2c75545059d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:22.235206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 360/706] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:39:22.286413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82918d43-0817-40a9-a058-a2c75545059d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:22.313854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:22.327656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701f198-e751-4db0-8695-095b848d1db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:22.328345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701f198-e751-4db0-8695-095b848d1db3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:22.328626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701f198-e751-4db0-8695-095b848d1db3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:22.328895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701f198-e751-4db0-8695-095b848d1db3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:22.340180 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701f198-e751-4db0-8695-095b848d1db3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:22.341691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701f198-e751-4db0-8695-095b848d1db3 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-3701f198-e751-4db0-8695-095b848d1db3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:22.357707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:22.358652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:22.359519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:22.359519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:22.359519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:22.359519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 347/707] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:22.366643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009345d-226f-4be2-9ebd-3a99278e27ad None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:22.482423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009345d-226f-4be2-9ebd-3a99278e27ad None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:22.523927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2596f67-66b1-488b-bfe1-e0b1628ee65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:22.524548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2596f67-66b1-488b-bfe1-e0b1628ee65b None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:22.524787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2596f67-66b1-488b-bfe1-e0b1628ee65b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:22.525056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2596f67-66b1-488b-bfe1-e0b1628ee65b None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:22.525716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2596f67-66b1-488b-bfe1-e0b1628ee65b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:22.526014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2596f67-66b1-488b-bfe1-e0b1628ee65b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:22.527364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2596f67-66b1-488b-bfe1-e0b1628ee65b None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:22.544822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 361/708] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:22.550091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701f198-e751-4db0-8695-095b848d1db3 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:22.576986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2596f67-66b1-488b-bfe1-e0b1628ee65b None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:22.580787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:22.723138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701f198-e751-4db0-8695-095b848d1db3 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:22.757247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2596f67-66b1-488b-bfe1-e0b1628ee65b None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:22.783195 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de28685-3ea8-4a88-8e85-7b1ee38372f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:22.783825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de28685-3ea8-4a88-8e85-7b1ee38372f0 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:22.784100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de28685-3ea8-4a88-8e85-7b1ee38372f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:22.784362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de28685-3ea8-4a88-8e85-7b1ee38372f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:22.796067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de28685-3ea8-4a88-8e85-7b1ee38372f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:22.797573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de28685-3ea8-4a88-8e85-7b1ee38372f0 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:22.808786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122bf5f-2fcd-40f9-88cc-88fd19ffff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:22.809420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122bf5f-2fcd-40f9-88cc-88fd19ffff0e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:22.809724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122bf5f-2fcd-40f9-88cc-88fd19ffff0e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:22.809993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122bf5f-2fcd-40f9-88cc-88fd19ffff0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-7de28685-3ea8-4a88-8e85-7b1ee38372f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:22.811946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:22.812775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:22.813662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:22.813662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:22.813662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:22.813662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:22.813662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:22.813662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:22.813662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:22.813662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:22.815980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 348/709] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:22.817360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e7b7e-02de-46e4-8f02-866db2edffb1 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:22.817811 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e7b7e-02de-46e4-8f02-866db2edffb1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:22.818238 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e7b7e-02de-46e4-8f02-866db2edffb1 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:22.820488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 349/710] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:22.821403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e7b7e-02de-46e4-8f02-866db2edffb1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:22.828225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122bf5f-2fcd-40f9-88cc-88fd19ffff0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:22.829849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122bf5f-2fcd-40f9-88cc-88fd19ffff0e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:22.843029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-5122bf5f-2fcd-40f9-88cc-88fd19ffff0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:22.845680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:22.849087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:22.850296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:22.850296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:22.850296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:22.850296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 362/711] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:22.850714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0500-245c-4e2b-9130-8fb4ea8ee584 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:22.851099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0500-245c-4e2b-9130-8fb4ea8ee584 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:22.851486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0500-245c-4e2b-9130-8fb4ea8ee584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:22.871187 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-7d5e0500-245c-4e2b-9130-8fb4ea8ee584 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:22.872514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7d5e0500-245c-4e2b-9130-8fb4ea8ee584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:22.937927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:22.950112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e7b7e-02de-46e4-8f02-866db2edffb1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:22.961574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 363/712] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:22.962769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e0500-245c-4e2b-9130-8fb4ea8ee584 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:23.019301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21645b2-6138-4bb3-b7ea-69295b9a4efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:23.021205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21645b2-6138-4bb3-b7ea-69295b9a4efa None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:23.022853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21645b2-6138-4bb3-b7ea-69295b9a4efa None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:23.023544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21645b2-6138-4bb3-b7ea-69295b9a4efa None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:23.025315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21645b2-6138-4bb3-b7ea-69295b9a4efa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:23.026052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21645b2-6138-4bb3-b7ea-69295b9a4efa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:23.028184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21645b2-6138-4bb3-b7ea-69295b9a4efa None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:23.054381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 350/713] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:23.060439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21645b2-6138-4bb3-b7ea-69295b9a4efa None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:23.156789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e0500-245c-4e2b-9130-8fb4ea8ee584 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:23.193297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:23.210926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521148a-a41a-4ec1-8956-87bc7c17cffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:23.211852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521148a-a41a-4ec1-8956-87bc7c17cffc None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:23.212453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521148a-a41a-4ec1-8956-87bc7c17cffc None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:23.212935 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521148a-a41a-4ec1-8956-87bc7c17cffc None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:23.213802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521148a-a41a-4ec1-8956-87bc7c17cffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:23.214299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521148a-a41a-4ec1-8956-87bc7c17cffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:23.215960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521148a-a41a-4ec1-8956-87bc7c17cffc None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:23.232727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 364/714] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:23.238082 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21645b2-6138-4bb3-b7ea-69295b9a4efa None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:23.239379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521148a-a41a-4ec1-8956-87bc7c17cffc None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:23.307927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91544f0b-7061-4bc3-8745-b81df604bbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:23.308812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91544f0b-7061-4bc3-8745-b81df604bbcf None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:23.309315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91544f0b-7061-4bc3-8745-b81df604bbcf None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:23.309825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91544f0b-7061-4bc3-8745-b81df604bbcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:23.310774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91544f0b-7061-4bc3-8745-b81df604bbcf None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=1a4a95ae5ee84f259ffcd7528b0fa505) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:23.311295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91544f0b-7061-4bc3-8745-b81df604bbcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:23.312948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91544f0b-7061-4bc3-8745-b81df604bbcf None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:23.319077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 351/715] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:23 2022] GET /identity/v3/role_assignments?user.id=1a4a95ae5ee84f259ffcd7528b0fa505&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:23.319344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:23.320381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91544f0b-7061-4bc3-8745-b81df604bbcf None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:23.384757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521148a-a41a-4ec1-8956-87bc7c17cffc None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:23.426099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26d4ff-e06a-4798-bd08-0b26aafce4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:23.426768 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26d4ff-e06a-4798-bd08-0b26aafce4ee None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:23.427050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26d4ff-e06a-4798-bd08-0b26aafce4ee None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:23.427567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26d4ff-e06a-4798-bd08-0b26aafce4ee None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:23.428238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26d4ff-e06a-4798-bd08-0b26aafce4ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:23.431052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26d4ff-e06a-4798-bd08-0b26aafce4ee None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:23.432566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91544f0b-7061-4bc3-8745-b81df604bbcf None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:23.469267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 365/716] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:39:23 2022] POST /identity/v3/users => generated 254 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:39:23.489799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ce0b4-0283-4e44-a972-0f7ac612901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:23.490858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ce0b4-0283-4e44-a972-0f7ac612901a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:23.491868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ce0b4-0283-4e44-a972-0f7ac612901a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:23.492384 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ce0b4-0283-4e44-a972-0f7ac612901a None admin] PATH_INFO: `/v3/services/network` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:23.493224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ce0b4-0283-4e44-a972-0f7ac612901a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:23.499294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ce0b4-0283-4e44-a972-0f7ac612901a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-4b3ce0b4-0283-4e44-a972-0f7ac612901a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:23.518259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:23.519374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:23.520229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 352/717] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:23 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:23.528725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26d4ff-e06a-4798-bd08-0b26aafce4ee None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:23.684369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26d4ff-e06a-4798-bd08-0b26aafce4ee None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:23.686075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:23.724972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55825362-7615-4af2-99df-c34cd4ed75b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:23.726146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55825362-7615-4af2-99df-c34cd4ed75b3 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:23.726419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55825362-7615-4af2-99df-c34cd4ed75b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:23.726682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55825362-7615-4af2-99df-c34cd4ed75b3 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:23.727311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55825362-7615-4af2-99df-c34cd4ed75b3 None admin] RBAC: Adding query filter params (name=network) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:23.727604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55825362-7615-4af2-99df-c34cd4ed75b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:23.728975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55825362-7615-4af2-99df-c34cd4ed75b3 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:23.733998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 366/718] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:23.739677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ce0b4-0283-4e44-a972-0f7ac612901a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:23.751525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:23.852725 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ce0b4-0283-4e44-a972-0f7ac612901a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:23.891376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4f8f1-4bbb-4604-ba76-27cf7c3c8e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:23.891986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4f8f1-4bbb-4604-ba76-27cf7c3c8e0e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:23.892264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4f8f1-4bbb-4604-ba76-27cf7c3c8e0e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:23.892533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4f8f1-4bbb-4604-ba76-27cf7c3c8e0e None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:23.893177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4f8f1-4bbb-4604-ba76-27cf7c3c8e0e None admin] RBAC: Adding query filter params (type=network) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:23.893475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4f8f1-4bbb-4604-ba76-27cf7c3c8e0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:23.895044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4f8f1-4bbb-4604-ba76-27cf7c3c8e0e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:23.901932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 353/719] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:23.978810 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf78136-ad8c-49e0-bacc-b6e468a95148 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:23.979395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf78136-ad8c-49e0-bacc-b6e468a95148 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:23.979850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf78136-ad8c-49e0-bacc-b6e468a95148 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:23.981854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 367/720] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:23.987619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8971451-4c1d-460d-9f97-ca89fb8b994b None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:23.988052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8971451-4c1d-460d-9f97-ca89fb8b994b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:23.988422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8971451-4c1d-460d-9f97-ca89fb8b994b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:24.016460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-c8971451-4c1d-460d-9f97-ca89fb8b994b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:24.017707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c8971451-4c1d-460d-9f97-ca89fb8b994b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:24.112207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:24.119017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 354/721] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:24.416668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf78136-ad8c-49e0-bacc-b6e468a95148 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:24.537591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf78136-ad8c-49e0-bacc-b6e468a95148 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:24.573651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c286d-1852-4916-b401-a6490a1d3894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:24.574198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c286d-1852-4916-b401-a6490a1d3894 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:24.574474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c286d-1852-4916-b401-a6490a1d3894 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:24.574741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c286d-1852-4916-b401-a6490a1d3894 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:24.585671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c286d-1852-4916-b401-a6490a1d3894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-726c286d-1852-4916-b401-a6490a1d3894 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:24.590596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:24.591515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:24.592350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 368/722] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:24.597482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8971451-4c1d-460d-9f97-ca89fb8b994b None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:24.756622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8971451-4c1d-460d-9f97-ca89fb8b994b None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:24.800551 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caebf5-adb7-456f-b67c-ee5598bfcf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:24.801172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caebf5-adb7-456f-b67c-ee5598bfcf0f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:24.801455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caebf5-adb7-456f-b67c-ee5598bfcf0f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:24.801754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caebf5-adb7-456f-b67c-ee5598bfcf0f None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:24.802501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caebf5-adb7-456f-b67c-ee5598bfcf0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:24.802811 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caebf5-adb7-456f-b67c-ee5598bfcf0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:24.804737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caebf5-adb7-456f-b67c-ee5598bfcf0f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:24.816878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 355/723] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:24.822669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c286d-1852-4916-b401-a6490a1d3894 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:24.863456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:24.892030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785152a6-8300-4094-9e7c-4a316be1846d None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:24.892337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785152a6-8300-4094-9e7c-4a316be1846d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:24.892697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785152a6-8300-4094-9e7c-4a316be1846d None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:24.894396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 356/724] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:24.900354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593d521-5458-489c-9393-ac27981aa4fd None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:24.900674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593d521-5458-489c-9393-ac27981aa4fd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:24.900920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593d521-5458-489c-9393-ac27981aa4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:24.930736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b593d521-5458-489c-9393-ac27981aa4fd None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:24.932131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b593d521-5458-489c-9393-ac27981aa4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:24.950863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c286d-1852-4916-b401-a6490a1d3894 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:24.998060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14be907-f28b-48c0-928e-97ab3decb931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:24.998757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14be907-f28b-48c0-928e-97ab3decb931 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:24.999037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14be907-f28b-48c0-928e-97ab3decb931 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:24.999331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14be907-f28b-48c0-928e-97ab3decb931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:25.013980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:25.015767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14be907-f28b-48c0-928e-97ab3decb931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:25.017292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14be907-f28b-48c0-928e-97ab3decb931 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:25.027492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 357/725] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-d14be907-f28b-48c0-928e-97ab3decb931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:25.041095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:25.042079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:25.042950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:25.042950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:25.042950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:25.045190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 369/726] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:25.050445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593d521-5458-489c-9393-ac27981aa4fd None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:25.198993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593d521-5458-489c-9393-ac27981aa4fd None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:25.244860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1033b-fd9f-4b07-af75-a611ab53844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:25.245470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1033b-fd9f-4b07-af75-a611ab53844b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:25.245770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1033b-fd9f-4b07-af75-a611ab53844b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:25.246041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1033b-fd9f-4b07-af75-a611ab53844b None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:25.246685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1033b-fd9f-4b07-af75-a611ab53844b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:25.246986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1033b-fd9f-4b07-af75-a611ab53844b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:25.248341 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1033b-fd9f-4b07-af75-a611ab53844b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:25.260334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 358/727] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:25.265469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14be907-f28b-48c0-928e-97ab3decb931 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:25.325864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1033b-fd9f-4b07-af75-a611ab53844b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:25.368548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:25.388092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14be907-f28b-48c0-928e-97ab3decb931 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:25.419726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:25.425929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23489990-7892-4b3a-93b3-5258beb6f594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:25.427035 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23489990-7892-4b3a-93b3-5258beb6f594 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:25.427536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23489990-7892-4b3a-93b3-5258beb6f594 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:25.427997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23489990-7892-4b3a-93b3-5258beb6f594 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:25.436301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23489990-7892-4b3a-93b3-5258beb6f594 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:25.438044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23489990-7892-4b3a-93b3-5258beb6f594 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-23489990-7892-4b3a-93b3-5258beb6f594 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:25.451828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:25.452749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:25.453700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:25.453700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:25.453700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:25.453700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:25.453700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:25.453700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:25.453700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:25.453700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:25.453700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:25.453700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:25.456611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 370/728] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:25.462301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23489990-7892-4b3a-93b3-5258beb6f594 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:25.510533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1033b-fd9f-4b07-af75-a611ab53844b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:25.547684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c4ac0-700b-413e-b930-8a01767aaab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:25.548666 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c4ac0-700b-413e-b930-8a01767aaab9 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:25.549140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c4ac0-700b-413e-b930-8a01767aaab9 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:25.549641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c4ac0-700b-413e-b930-8a01767aaab9 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:25.550514 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c4ac0-700b-413e-b930-8a01767aaab9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:25.552089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c4ac0-700b-413e-b930-8a01767aaab9 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:25.571390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 359/729] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:39:25 2022] POST /identity/v3/services => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:39:25.639649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23489990-7892-4b3a-93b3-5258beb6f594 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:25.685304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604d7c4-f3f5-4a6a-9ea1-ff77106f58a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:25.686349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604d7c4-f3f5-4a6a-9ea1-ff77106f58a2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:25.686823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604d7c4-f3f5-4a6a-9ea1-ff77106f58a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:25.687316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604d7c4-f3f5-4a6a-9ea1-ff77106f58a2 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:25.688239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604d7c4-f3f5-4a6a-9ea1-ff77106f58a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:25.688755 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604d7c4-f3f5-4a6a-9ea1-ff77106f58a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:25.690479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604d7c4-f3f5-4a6a-9ea1-ff77106f58a2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:25.692586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:25.712220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 371/730] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 3 14:39:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:25.717452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c4ac0-700b-413e-b930-8a01767aaab9 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:25.828575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c4ac0-700b-413e-b930-8a01767aaab9 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:25.844069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:25.869374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5982375-b819-495a-bc9b-b447c3eb3406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:25.870360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5982375-b819-495a-bc9b-b447c3eb3406 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:25.870884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5982375-b819-495a-bc9b-b447c3eb3406 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:25.871353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5982375-b819-495a-bc9b-b447c3eb3406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:25.940354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5982375-b819-495a-bc9b-b447c3eb3406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:25.942250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5982375-b819-495a-bc9b-b447c3eb3406 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:26.052101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 360/731] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:25 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 14:39:26.062385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604d7c4-f3f5-4a6a-9ea1-ff77106f58a2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:26.087574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db33a79-2865-420d-a0df-cce6a96b3a8d None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.087886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db33a79-2865-420d-a0df-cce6a96b3a8d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.088121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db33a79-2865-420d-a0df-cce6a96b3a8d None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.089767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 361/732] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:26.094683 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d531c-e1d3-4d55-917b-c4c19ea2fb17 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.095125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d531c-e1d3-4d55-917b-c4c19ea2fb17 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.095537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d531c-e1d3-4d55-917b-c4c19ea2fb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.116117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d27d531c-e1d3-4d55-917b-c4c19ea2fb17 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:26.117579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d27d531c-e1d3-4d55-917b-c4c19ea2fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:26.129990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:26.190921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 362/733] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 14:39:26.232604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604d7c4-f3f5-4a6a-9ea1-ff77106f58a2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:26.294398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b912629-b185-4dd2-a596-82277b68e3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:26.294666 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b912629-b185-4dd2-a596-82277b68e3fb None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.294915 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b912629-b185-4dd2-a596-82277b68e3fb None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.295188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b912629-b185-4dd2-a596-82277b68e3fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.322408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b912629-b185-4dd2-a596-82277b68e3fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:26.323796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b912629-b185-4dd2-a596-82277b68e3fb None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-7b912629-b185-4dd2-a596-82277b68e3fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:26.346907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:26.347889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:26.349366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:26.349366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:26.349366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:26.349366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 372/734] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:26.354143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d531c-e1d3-4d55-917b-c4c19ea2fb17 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:26.482208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d531c-e1d3-4d55-917b-c4c19ea2fb17 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:26.492648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b912629-b185-4dd2-a596-82277b68e3fb None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:26.527323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bb987-121d-4466-bc7c-def9ed667683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:26.527935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bb987-121d-4466-bc7c-def9ed667683 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.528209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bb987-121d-4466-bc7c-def9ed667683 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.528477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bb987-121d-4466-bc7c-def9ed667683 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.529128 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bb987-121d-4466-bc7c-def9ed667683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:26.529428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bb987-121d-4466-bc7c-def9ed667683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:26.533529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bb987-121d-4466-bc7c-def9ed667683 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:26.544593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 363/735] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:26.550531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bb987-121d-4466-bc7c-def9ed667683 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:26.554698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:26.599659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:26.667070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b912629-b185-4dd2-a596-82277b68e3fb None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:26.708822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9eb039-ca4d-43e8-97ce-111b444c5003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:26.709456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eb039-ca4d-43e8-97ce-111b444c5003 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.709801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eb039-ca4d-43e8-97ce-111b444c5003 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.710073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eb039-ca4d-43e8-97ce-111b444c5003 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.719521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bb987-121d-4466-bc7c-def9ed667683 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:26.720604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9eb039-ca4d-43e8-97ce-111b444c5003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-cc9eb039-ca4d-43e8-97ce-111b444c5003 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:26.727017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:26.728033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:26.728859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 373/736] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:26.730543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58785040-aa6a-40f3-9d82-734937026a03 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.730543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58785040-aa6a-40f3-9d82-734937026a03 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.730543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58785040-aa6a-40f3-9d82-734937026a03 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.735563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 374/737] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:26.735563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d841ba1-8ab6-406f-b6cf-9a49a422ac55 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.735563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d841ba1-8ab6-406f-b6cf-9a49a422ac55 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.735563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d841ba1-8ab6-406f-b6cf-9a49a422ac55 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.736229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 375/738] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:26.737865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d841ba1-8ab6-406f-b6cf-9a49a422ac55 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:26.762836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e10a80-42eb-48b7-aa78-2f8dbb2974fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:26.763654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e10a80-42eb-48b7-aa78-2f8dbb2974fc None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.763941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e10a80-42eb-48b7-aa78-2f8dbb2974fc None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.764209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e10a80-42eb-48b7-aa78-2f8dbb2974fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.773453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e10a80-42eb-48b7-aa78-2f8dbb2974fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:26.774975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e10a80-42eb-48b7-aa78-2f8dbb2974fc None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-c6e10a80-42eb-48b7-aa78-2f8dbb2974fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:26.789107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:26.790032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:26.790969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:26.790969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:26.790969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:26.790969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:26.790969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:26.790969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:26.790969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:26.790969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:26.792077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 364/739] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:26.793331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c5190-6f62-466b-972b-c4028cd76903 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.793670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c5190-6f62-466b-972b-c4028cd76903 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.793952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c5190-6f62-466b-972b-c4028cd76903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.811835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-279c5190-6f62-466b-972b-c4028cd76903 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:26.813291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-279c5190-6f62-466b-972b-c4028cd76903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:26.848454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d841ba1-8ab6-406f-b6cf-9a49a422ac55 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:26.850808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:26.878030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:26.885199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524140b8-18f6-4686-bddd-efa54e2f48a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:26.885850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524140b8-18f6-4686-bddd-efa54e2f48a3 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.886131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524140b8-18f6-4686-bddd-efa54e2f48a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.886406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524140b8-18f6-4686-bddd-efa54e2f48a3 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.887082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524140b8-18f6-4686-bddd-efa54e2f48a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:26.887385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524140b8-18f6-4686-bddd-efa54e2f48a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:26.888753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524140b8-18f6-4686-bddd-efa54e2f48a3 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:26.901547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 376/740] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:26.901547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c2eea-229e-463c-a9be-17ab174ef317 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:26.901547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c2eea-229e-463c-a9be-17ab174ef317 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:26.901547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c2eea-229e-463c-a9be-17ab174ef317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:26.902402 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 365/741] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:26.903749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c5190-6f62-466b-972b-c4028cd76903 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:26.921097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c59c2eea-229e-463c-a9be-17ab174ef317 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:26.923049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c59c2eea-229e-463c-a9be-17ab174ef317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:27.010515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c5190-6f62-466b-972b-c4028cd76903 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:27.012381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 377/742] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 14:39:27.013429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c2eea-229e-463c-a9be-17ab174ef317 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:27.094231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5275b09-2ba6-4f13-ace5-cc48c5b4356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:27.094866 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5275b09-2ba6-4f13-ace5-cc48c5b4356f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:27.095154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5275b09-2ba6-4f13-ace5-cc48c5b4356f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:27.095430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5275b09-2ba6-4f13-ace5-cc48c5b4356f None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:27.096139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5275b09-2ba6-4f13-ace5-cc48c5b4356f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:27.096446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5275b09-2ba6-4f13-ace5-cc48c5b4356f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:27.098030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5275b09-2ba6-4f13-ace5-cc48c5b4356f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:27.119003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c2eea-229e-463c-a9be-17ab174ef317 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:27.121860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:27.125948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 366/743] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:27.136357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5275b09-2ba6-4f13-ace5-cc48c5b4356f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:27.178531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9548e1-8ffb-430f-b673-bc521ff81abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:27.179208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9548e1-8ffb-430f-b673-bc521ff81abc None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:27.179483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9548e1-8ffb-430f-b673-bc521ff81abc None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:27.179748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9548e1-8ffb-430f-b673-bc521ff81abc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:27.192633 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9548e1-8ffb-430f-b673-bc521ff81abc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:27.194144 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9548e1-8ffb-430f-b673-bc521ff81abc None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-3f9548e1-8ffb-430f-b673-bc521ff81abc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:27.208876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:27.210602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:27.211711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:27.211711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:27.211711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:27.215243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 378/744] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:27.216070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9548e1-8ffb-430f-b673-bc521ff81abc None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:27.219495 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:27.289025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5275b09-2ba6-4f13-ace5-cc48c5b4356f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:27.323654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9548e1-8ffb-430f-b673-bc521ff81abc None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:27.333422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100e85e-fa52-42e8-916a-a6ed3b4bb726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:27.334504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100e85e-fa52-42e8-916a-a6ed3b4bb726 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:27.335319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100e85e-fa52-42e8-916a-a6ed3b4bb726 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:27.335985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100e85e-fa52-42e8-916a-a6ed3b4bb726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:27.337293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100e85e-fa52-42e8-916a-a6ed3b4bb726 None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=6fbf23ae20ee4f6c979e256d2d4daa5f) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:27.337818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100e85e-fa52-42e8-916a-a6ed3b4bb726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:27.341246 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100e85e-fa52-42e8-916a-a6ed3b4bb726 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:27.348578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 367/745] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:27 2022] GET /identity/v3/role_assignments?user.id=6fbf23ae20ee4f6c979e256d2d4daa5f&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:27.349850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100e85e-fa52-42e8-916a-a6ed3b4bb726 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:27.360201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e59c7-1404-4321-86dd-e37c7831a2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:27.360942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e59c7-1404-4321-86dd-e37c7831a2e8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:27.361216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e59c7-1404-4321-86dd-e37c7831a2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:27.361483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e59c7-1404-4321-86dd-e37c7831a2e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:27.374577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e59c7-1404-4321-86dd-e37c7831a2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-e59e59c7-1404-4321-86dd-e37c7831a2e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:27.379894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:27.380953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:27.381966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 379/746] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:27.382654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e59c7-1404-4321-86dd-e37c7831a2e8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:27.398590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:27.473178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100e85e-fa52-42e8-916a-a6ed3b4bb726 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:27.496875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:27.511300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e59c7-1404-4321-86dd-e37c7831a2e8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:27.521320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da01d6-b188-4edc-a4d5-24b2709ae39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:27.522091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da01d6-b188-4edc-a4d5-24b2709ae39d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:27.522455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da01d6-b188-4edc-a4d5-24b2709ae39d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:27.522737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da01d6-b188-4edc-a4d5-24b2709ae39d None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:27.523511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da01d6-b188-4edc-a4d5-24b2709ae39d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:27.523818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da01d6-b188-4edc-a4d5-24b2709ae39d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:27.525342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da01d6-b188-4edc-a4d5-24b2709ae39d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:27.542594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 368/747] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:27.542594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da01d6-b188-4edc-a4d5-24b2709ae39d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:27.555288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ce9eb-56be-46ae-8d20-491a060f05de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:27.556215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ce9eb-56be-46ae-8d20-491a060f05de None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:27.556491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ce9eb-56be-46ae-8d20-491a060f05de None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:27.556759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ce9eb-56be-46ae-8d20-491a060f05de None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:27.557407 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ce9eb-56be-46ae-8d20-491a060f05de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:27.558812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ce9eb-56be-46ae-8d20-491a060f05de None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:27.578131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 380/748] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:39:27 2022] POST /identity/v3/services => generated 240 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:39:27.579377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ce9eb-56be-46ae-8d20-491a060f05de None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:27.673969 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:27.682635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da01d6-b188-4edc-a4d5-24b2709ae39d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:27.720346 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ce9eb-56be-46ae-8d20-491a060f05de None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:27.724726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40106c-f605-4460-bf91-27fd2d61ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:27.725512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40106c-f605-4460-bf91-27fd2d61ff30 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:27.725903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40106c-f605-4460-bf91-27fd2d61ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:27.726297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40106c-f605-4460-bf91-27fd2d61ff30 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:27.727139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40106c-f605-4460-bf91-27fd2d61ff30 None admin] RBAC: Adding query filter params (name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:27.727596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40106c-f605-4460-bf91-27fd2d61ff30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:27.729159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40106c-f605-4460-bf91-27fd2d61ff30 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:27.739211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 369/749] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:27.746735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40106c-f605-4460-bf91-27fd2d61ff30 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:27.770240 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b086914-9668-46dc-ba18-0b6e66fd2cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:27.771380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b086914-9668-46dc-ba18-0b6e66fd2cad None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:27.771876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b086914-9668-46dc-ba18-0b6e66fd2cad None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:27.772978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b086914-9668-46dc-ba18-0b6e66fd2cad None admin] PATH_INFO: `/v3/users/nova` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:27.789065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b086914-9668-46dc-ba18-0b6e66fd2cad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:27.791462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b086914-9668-46dc-ba18-0b6e66fd2cad None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-6b086914-9668-46dc-ba18-0b6e66fd2cad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:27.815577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:27.816586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:27.817544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:27.817544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:27.817544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:27.817544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:27.817544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:27.817544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:27.817544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:27.817544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:27.817544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:27.817544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:27.823419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 381/750] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 3 14:39:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:27.827104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b086914-9668-46dc-ba18-0b6e66fd2cad None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:27.837091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:27.950548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b086914-9668-46dc-ba18-0b6e66fd2cad None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:27.958958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40106c-f605-4460-bf91-27fd2d61ff30 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:27.993320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e0178-baee-4860-b094-620742803c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:27.994005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e0178-baee-4860-b094-620742803c1e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:27.994284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e0178-baee-4860-b094-620742803c1e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:27.994559 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e0178-baee-4860-b094-620742803c1e None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:27.995421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e0178-baee-4860-b094-620742803c1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:27.995987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e0178-baee-4860-b094-620742803c1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:27.999792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e0178-baee-4860-b094-620742803c1e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:28.002094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:28.008623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea246e-7a50-4159-a7df-efeaedac520b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:28.009332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea246e-7a50-4159-a7df-efeaedac520b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:28.009673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea246e-7a50-4159-a7df-efeaedac520b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:28.009931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea246e-7a50-4159-a7df-efeaedac520b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:28.021721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea246e-7a50-4159-a7df-efeaedac520b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:28.023212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea246e-7a50-4159-a7df-efeaedac520b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:28.026431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 382/751] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 3 14:39:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:28.034460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e0178-baee-4860-b094-620742803c1e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-45ea246e-7a50-4159-a7df-efeaedac520b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:28.041762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:28.042686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:28.043569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:28.043569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:28.043569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:28.043569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 370/752] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:28.048593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea246e-7a50-4159-a7df-efeaedac520b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:28.142774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:28.153923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea246e-7a50-4159-a7df-efeaedac520b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:28.206335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec40d3-aef5-42aa-a84e-1d72a5318361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:28.207293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec40d3-aef5-42aa-a84e-1d72a5318361 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:28.207790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec40d3-aef5-42aa-a84e-1d72a5318361 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:28.208266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec40d3-aef5-42aa-a84e-1d72a5318361 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:28.208998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec40d3-aef5-42aa-a84e-1d72a5318361 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:28.209371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec40d3-aef5-42aa-a84e-1d72a5318361 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:28.210899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec40d3-aef5-42aa-a84e-1d72a5318361 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:28.231800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 371/753] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:28.235309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e0178-baee-4860-b094-620742803c1e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:28.237489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec40d3-aef5-42aa-a84e-1d72a5318361 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:28.275274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c22dcc-9ac4-401a-8c7f-5259a8c7f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:28.282159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c22dcc-9ac4-401a-8c7f-5259a8c7f1ff None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:28.282819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c22dcc-9ac4-401a-8c7f-5259a8c7f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:28.283382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c22dcc-9ac4-401a-8c7f-5259a8c7f1ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:28.302683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c22dcc-9ac4-401a-8c7f-5259a8c7f1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:28.305551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c22dcc-9ac4-401a-8c7f-5259a8c7f1ff None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-54c22dcc-9ac4-401a-8c7f-5259a8c7f1ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:28.328575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:28.329737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:28.330625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:28.330625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:28.330625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:28.335822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 383/754] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:28.335822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b122295-d8fd-4437-b493-12a0659c0847 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:28.335822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b122295-d8fd-4437-b493-12a0659c0847 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:28.335822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b122295-d8fd-4437-b493-12a0659c0847 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:28.338092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 384/755] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:28.342838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b122295-d8fd-4437-b493-12a0659c0847 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:28.380096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:28.418690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec40d3-aef5-42aa-a84e-1d72a5318361 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:28.457577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb5252-4235-42c2-bd4c-bbe1db50906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:28.458392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb5252-4235-42c2-bd4c-bbe1db50906f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:28.458682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb5252-4235-42c2-bd4c-bbe1db50906f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:28.458960 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb5252-4235-42c2-bd4c-bbe1db50906f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:28.466458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb5252-4235-42c2-bd4c-bbe1db50906f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:28.467886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb5252-4235-42c2-bd4c-bbe1db50906f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:28.471819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b122295-d8fd-4437-b493-12a0659c0847 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-63cb5252-4235-42c2-bd4c-bbe1db50906f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:28.481079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:28.482350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:28.483354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 372/756] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 3 14:39:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:28.483925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da951ce-030c-4918-a01a-7a4471937cdc None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:28.483999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da951ce-030c-4918-a01a-7a4471937cdc None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:28.484585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da951ce-030c-4918-a01a-7a4471937cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:28.503525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2da951ce-030c-4918-a01a-7a4471937cdc None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:28.504685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2da951ce-030c-4918-a01a-7a4471937cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:28.508814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:28.519650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88175cc-daf3-4746-85bd-113bdc008880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:28.520602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88175cc-daf3-4746-85bd-113bdc008880 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:28.521082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88175cc-daf3-4746-85bd-113bdc008880 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:28.522002 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88175cc-daf3-4746-85bd-113bdc008880 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:28.522925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88175cc-daf3-4746-85bd-113bdc008880 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:28.523699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88175cc-daf3-4746-85bd-113bdc008880 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:28.525288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88175cc-daf3-4746-85bd-113bdc008880 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:28.542440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 385/757] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:28.543776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88175cc-daf3-4746-85bd-113bdc008880 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:28.596990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 373/758] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:28.598293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da951ce-030c-4918-a01a-7a4471937cdc None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:28.658096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:28.666286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88175cc-daf3-4746-85bd-113bdc008880 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:28.705659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0e1d9-a70a-43d5-8862-38c91911bb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:28.707067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0e1d9-a70a-43d5-8862-38c91911bb92 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:28.707576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0e1d9-a70a-43d5-8862-38c91911bb92 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:28.708579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0e1d9-a70a-43d5-8862-38c91911bb92 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:28.710082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0e1d9-a70a-43d5-8862-38c91911bb92 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:28.710594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0e1d9-a70a-43d5-8862-38c91911bb92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:28.712250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0e1d9-a70a-43d5-8862-38c91911bb92 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:28.733486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 386/759] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 3 14:39:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:28.742093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0e1d9-a70a-43d5-8862-38c91911bb92 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:28.835910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da951ce-030c-4918-a01a-7a4471937cdc None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:28.878421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0e1d9-a70a-43d5-8862-38c91911bb92 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:28.900924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee8a41-e1d0-4d64-be45-f5f8d587ded7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:28.901578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee8a41-e1d0-4d64-be45-f5f8d587ded7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:28.901865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee8a41-e1d0-4d64-be45-f5f8d587ded7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:28.902138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee8a41-e1d0-4d64-be45-f5f8d587ded7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:28.915082 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee8a41-e1d0-4d64-be45-f5f8d587ded7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:28.916543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee8a41-e1d0-4d64-be45-f5f8d587ded7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:28.922120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-6bee8a41-e1d0-4d64-be45-f5f8d587ded7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:28.935533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:28.936623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:28.937566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:28.937566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:28.937566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:28.937566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:28.937566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:28.937566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:28.937566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:28.937566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:28.937566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 374/760] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:28.942405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee8a41-e1d0-4d64-be45-f5f8d587ded7 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:28.947161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c877602-b7d4-4cc9-b352-66c12c879722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:28.947779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c877602-b7d4-4cc9-b352-66c12c879722 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:28.948058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c877602-b7d4-4cc9-b352-66c12c879722 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:28.948327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c877602-b7d4-4cc9-b352-66c12c879722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:28.971594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:29.032591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c877602-b7d4-4cc9-b352-66c12c879722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:29.034652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c877602-b7d4-4cc9-b352-66c12c879722 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:29.073346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee8a41-e1d0-4d64-be45-f5f8d587ded7 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:29.121145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150174e8-3cac-4ec1-8752-c51ff7ac4131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:29.122187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150174e8-3cac-4ec1-8752-c51ff7ac4131 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.122678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150174e8-3cac-4ec1-8752-c51ff7ac4131 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.123153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150174e8-3cac-4ec1-8752-c51ff7ac4131 None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.124054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150174e8-3cac-4ec1-8752-c51ff7ac4131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:29.125046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150174e8-3cac-4ec1-8752-c51ff7ac4131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:29.127229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150174e8-3cac-4ec1-8752-c51ff7ac4131 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:29.145592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 375/761] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:29.146721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150174e8-3cac-4ec1-8752-c51ff7ac4131 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:29.157110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 387/762] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 14:39:29.158234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c877602-b7d4-4cc9-b352-66c12c879722 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:29.216748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:29.233013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:29.271707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150174e8-3cac-4ec1-8752-c51ff7ac4131 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:29.298330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c877602-b7d4-4cc9-b352-66c12c879722 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:29.314399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cf118-be76-4a8f-aa32-56eb364f6a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:29.315295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cf118-be76-4a8f-aa32-56eb364f6a93 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.315725 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cf118-be76-4a8f-aa32-56eb364f6a93 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.316145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cf118-be76-4a8f-aa32-56eb364f6a93 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.317126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cf118-be76-4a8f-aa32-56eb364f6a93 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:29.319366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cf118-be76-4a8f-aa32-56eb364f6a93 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-b52cf118-be76-4a8f-aa32-56eb364f6a93 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:29.329293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:29.330500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 14:39:29.331354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:29.339213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 376/763] 10.222.0.46 () {56 vars in 1129 bytes} [Mon Oct 3 14:39:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:29.339213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cf118-be76-4a8f-aa32-56eb364f6a93 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:29.345754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201418c5-4b62-4409-a193-fcaed042a6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:29.346691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201418c5-4b62-4409-a193-fcaed042a6b8 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.347194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201418c5-4b62-4409-a193-fcaed042a6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.347667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201418c5-4b62-4409-a193-fcaed042a6b8 None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/1a4a95ae5ee84f259ffcd7528b0fa505/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.351517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201418c5-4b62-4409-a193-fcaed042a6b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=1a4a95ae5ee84f259ffcd7528b0fa505, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:29.353966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201418c5-4b62-4409-a193-fcaed042a6b8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:29.374736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 388/764] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:39:29 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/1a4a95ae5ee84f259ffcd7528b0fa505/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:39:29.376329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201418c5-4b62-4409-a193-fcaed042a6b8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:29.455238 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cf118-be76-4a8f-aa32-56eb364f6a93 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:29.484022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:29.497873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206abd2-1c88-4caf-b1d7-e27f8dcfafd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:29.498748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206abd2-1c88-4caf-b1d7-e27f8dcfafd9 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.499158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206abd2-1c88-4caf-b1d7-e27f8dcfafd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.499528 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206abd2-1c88-4caf-b1d7-e27f8dcfafd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.513099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206abd2-1c88-4caf-b1d7-e27f8dcfafd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:29.514781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206abd2-1c88-4caf-b1d7-e27f8dcfafd9 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:29.525056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201418c5-4b62-4409-a193-fcaed042a6b8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-4206abd2-1c88-4caf-b1d7-e27f8dcfafd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:29.530924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:29.532134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:29.533242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:29.533242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:29.533242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:29.534710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 377/765] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:29.540308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206abd2-1c88-4caf-b1d7-e27f8dcfafd9 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:29.543053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:29.578345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d91558-6bcc-440d-8e1b-6affdb25f6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:29.579597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d91558-6bcc-440d-8e1b-6affdb25f6a8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.580420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d91558-6bcc-440d-8e1b-6affdb25f6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.580865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d91558-6bcc-440d-8e1b-6affdb25f6a8 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.581848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d91558-6bcc-440d-8e1b-6affdb25f6a8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:29.582320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d91558-6bcc-440d-8e1b-6affdb25f6a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:29.585528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d91558-6bcc-440d-8e1b-6affdb25f6a8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:29.591591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 389/766] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 3 14:39:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:29.597177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d91558-6bcc-440d-8e1b-6affdb25f6a8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:29.667847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206abd2-1c88-4caf-b1d7-e27f8dcfafd9 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:29.707131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7504c5-6401-4447-965c-9930df5fac29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:29.707901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7504c5-6401-4447-965c-9930df5fac29 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.708506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7504c5-6401-4447-965c-9930df5fac29 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.708665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7504c5-6401-4447-965c-9930df5fac29 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.710100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7504c5-6401-4447-965c-9930df5fac29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:29.710508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7504c5-6401-4447-965c-9930df5fac29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:29.711967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7504c5-6401-4447-965c-9930df5fac29 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:29.726029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d91558-6bcc-440d-8e1b-6affdb25f6a8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:29.732776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 378/767] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:29.738361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7504c5-6401-4447-965c-9930df5fac29 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:29.764594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5475f69-2bc5-41db-99f2-9f80f95c68f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:29.765479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5475f69-2bc5-41db-99f2-9f80f95c68f4 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.765948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5475f69-2bc5-41db-99f2-9f80f95c68f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.766364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5475f69-2bc5-41db-99f2-9f80f95c68f4 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.767164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5475f69-2bc5-41db-99f2-9f80f95c68f4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:29.767615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5475f69-2bc5-41db-99f2-9f80f95c68f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:29.769131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5475f69-2bc5-41db-99f2-9f80f95c68f4 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:29.774708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 390/768] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 3 14:39:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:29.778768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:29.781289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5475f69-2bc5-41db-99f2-9f80f95c68f4 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:29.835734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:29.849333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7504c5-6401-4447-965c-9930df5fac29 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:29.886542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd8454-d5bd-4fc2-adbd-a5771fbd3637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:29.887180 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8454-d5bd-4fc2-adbd-a5771fbd3637 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.887462 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8454-d5bd-4fc2-adbd-a5771fbd3637 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.887737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8454-d5bd-4fc2-adbd-a5771fbd3637 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.894983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd8454-d5bd-4fc2-adbd-a5771fbd3637 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:29.896420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd8454-d5bd-4fc2-adbd-a5771fbd3637 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-54fd8454-d5bd-4fc2-adbd-a5771fbd3637 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:29.907984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:29.908893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:29.909829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:29.909829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:29.909829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:29.909829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:29.909829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:29.909829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:29.909829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:29.909829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:29.909829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 379/769] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:29.910779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7307-ebcb-4ce3-8b96-a89b5dd76a3f None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.910968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7307-ebcb-4ce3-8b96-a89b5dd76a3f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.911246 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7307-ebcb-4ce3-8b96-a89b5dd76a3f None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.913271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 380/770] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:29.913784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5475f69-2bc5-41db-99f2-9f80f95c68f4 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:29.920132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48110298-386e-421c-80f0-92d5ec16e623 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.920476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48110298-386e-421c-80f0-92d5ec16e623 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.920758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48110298-386e-421c-80f0-92d5ec16e623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.939055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-48110298-386e-421c-80f0-92d5ec16e623 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:29.940258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-48110298-386e-421c-80f0-92d5ec16e623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:29.954827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5f237-bbcf-4e55-a732-6b69e1208f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:29.955688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5f237-bbcf-4e55-a732-6b69e1208f89 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:29.956456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5f237-bbcf-4e55-a732-6b69e1208f89 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:29.956735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5f237-bbcf-4e55-a732-6b69e1208f89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:29.957750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5f237-bbcf-4e55-a732-6b69e1208f89 None admin] RBAC: Adding query filter params (interface=public, service_id=319eff679a0745b6bba244759498dc4d, region_id=RegionOne) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:29.958063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5f237-bbcf-4e55-a732-6b69e1208f89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:29.960210 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5f237-bbcf-4e55-a732-6b69e1208f89 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:29.970948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 391/771] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 3 14:39:29 2022] GET /identity/v3/endpoints?service_id=319eff679a0745b6bba244759498dc4d&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:29.972172 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5f237-bbcf-4e55-a732-6b69e1208f89 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:30.025751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 381/772] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:30.026815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48110298-386e-421c-80f0-92d5ec16e623 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:30.072131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:30.124839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5f237-bbcf-4e55-a732-6b69e1208f89 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:30.129441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:30.198024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1903b-c5b0-49af-bec7-b05a5277e3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:30.198768 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1903b-c5b0-49af-bec7-b05a5277e3e3 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:30.199058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1903b-c5b0-49af-bec7-b05a5277e3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:30.199328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1903b-c5b0-49af-bec7-b05a5277e3e3 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:30.200030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1903b-c5b0-49af-bec7-b05a5277e3e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:30.200335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1903b-c5b0-49af-bec7-b05a5277e3e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:30.201932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1903b-c5b0-49af-bec7-b05a5277e3e3 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:30.217318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 392/773] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:30.223489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1903b-c5b0-49af-bec7-b05a5277e3e3 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:30.226740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48110298-386e-421c-80f0-92d5ec16e623 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:30.277543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a176d8-5612-42c0-983e-daf463c45630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:30.278110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a176d8-5612-42c0-983e-daf463c45630 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:30.278391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a176d8-5612-42c0-983e-daf463c45630 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:30.278663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a176d8-5612-42c0-983e-daf463c45630 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:30.279299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a176d8-5612-42c0-983e-daf463c45630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:30.280736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a176d8-5612-42c0-983e-daf463c45630 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:30.296047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 382/774] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 3 14:39:30 2022] GET /identity/v3/services => generated 772 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:30.343737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a176d8-5612-42c0-983e-daf463c45630 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:30.428889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1903b-c5b0-49af-bec7-b05a5277e3e3 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:30.468904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c015c0-3cd1-4fa2-ac9b-90f99d649b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:30.470831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c015c0-3cd1-4fa2-ac9b-90f99d649b55 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:30.470831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c015c0-3cd1-4fa2-ac9b-90f99d649b55 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:30.470831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c015c0-3cd1-4fa2-ac9b-90f99d649b55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:30.470831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c015c0-3cd1-4fa2-ac9b-90f99d649b55 None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=fd0db9d2a753490cb8473229a8f6fe15) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:30.471129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c015c0-3cd1-4fa2-ac9b-90f99d649b55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:30.473097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c015c0-3cd1-4fa2-ac9b-90f99d649b55 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:30.474312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:30.481604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 393/775] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:30 2022] GET /identity/v3/role_assignments?user.id=fd0db9d2a753490cb8473229a8f6fe15&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:30.482855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fe104-c699-4c61-ad8d-3333790a2a95 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:30.483139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fe104-c699-4c61-ad8d-3333790a2a95 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:30.483410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fe104-c699-4c61-ad8d-3333790a2a95 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:30.484656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 394/776] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:30.492014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a176d8-5612-42c0-983e-daf463c45630 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:30.496248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e08903-a7d1-4f74-9c6e-d373e17fbc2a None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:30.496787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e08903-a7d1-4f74-9c6e-d373e17fbc2a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:30.497227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e08903-a7d1-4f74-9c6e-d373e17fbc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:30.520780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b9e08903-a7d1-4f74-9c6e-d373e17fbc2a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:30.522958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b9e08903-a7d1-4f74-9c6e-d373e17fbc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:30.538526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49e5b8-a1bd-4d80-8a13-7c6a976acf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:30.539198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49e5b8-a1bd-4d80-8a13-7c6a976acf87 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:30.539371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49e5b8-a1bd-4d80-8a13-7c6a976acf87 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:30.539655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49e5b8-a1bd-4d80-8a13-7c6a976acf87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:30.554739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:30.560869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49e5b8-a1bd-4d80-8a13-7c6a976acf87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-ef49e5b8-a1bd-4d80-8a13-7c6a976acf87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:30.566500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:30.567676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:30.569307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 383/777] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:30.574395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49e5b8-a1bd-4d80-8a13-7c6a976acf87 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:30.626583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 395/778] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:30.718554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49e5b8-a1bd-4d80-8a13-7c6a976acf87 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:30.762201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b0b05-5d7b-48df-9e2d-7f12d50fe4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:30.762813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b0b05-5d7b-48df-9e2d-7f12d50fe4b1 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:30.763092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b0b05-5d7b-48df-9e2d-7f12d50fe4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:30.763365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b0b05-5d7b-48df-9e2d-7f12d50fe4b1 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:30.764037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b0b05-5d7b-48df-9e2d-7f12d50fe4b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:30.764342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b0b05-5d7b-48df-9e2d-7f12d50fe4b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:30.765745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b0b05-5d7b-48df-9e2d-7f12d50fe4b1 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:30.774274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 384/779] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:30.779580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e08903-a7d1-4f74-9c6e-d373e17fbc2a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:30.897911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e08903-a7d1-4f74-9c6e-d373e17fbc2a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:30.947912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b0b05-5d7b-48df-9e2d-7f12d50fe4b1 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:30.948210 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f9bb1-2e3f-4e88-afe2-8080969dc31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:30.948882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f9bb1-2e3f-4e88-afe2-8080969dc31e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:30.948882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f9bb1-2e3f-4e88-afe2-8080969dc31e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:30.949050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f9bb1-2e3f-4e88-afe2-8080969dc31e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:30.969122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f9bb1-2e3f-4e88-afe2-8080969dc31e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:30.969846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:30.970704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f9bb1-2e3f-4e88-afe2-8080969dc31e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-5d2f9bb1-2e3f-4e88-afe2-8080969dc31e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:30.994426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:30.995482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:30.997201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:30.997201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:30.997201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:30.997201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 396/780] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:31.002423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f9bb1-2e3f-4e88-afe2-8080969dc31e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:31.025897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:31.069559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b0b05-5d7b-48df-9e2d-7f12d50fe4b1 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:31.108234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38f88b-3fd2-4227-8ba5-00bee699f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:31.109268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38f88b-3fd2-4227-8ba5-00bee699f238 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:31.109787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38f88b-3fd2-4227-8ba5-00bee699f238 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:31.110265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38f88b-3fd2-4227-8ba5-00bee699f238 None admin] PATH_INFO: `/v3/services/network` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:31.111057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38f88b-3fd2-4227-8ba5-00bee699f238 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:31.112761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38f88b-3fd2-4227-8ba5-00bee699f238 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-5c38f88b-3fd2-4227-8ba5-00bee699f238 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:31.123660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:31.124660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 14:39:31.125488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:31.132840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 385/781] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:30 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:31.134995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38f88b-3fd2-4227-8ba5-00bee699f238 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:31.231481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f9bb1-2e3f-4e88-afe2-8080969dc31e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:31.245824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38f88b-3fd2-4227-8ba5-00bee699f238 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:31.261944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:31.300506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4935aa4-db7d-4f9b-a67c-57677541a8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:31.301184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4935aa4-db7d-4f9b-a67c-57677541a8ef None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:31.301474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4935aa4-db7d-4f9b-a67c-57677541a8ef None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:31.301774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4935aa4-db7d-4f9b-a67c-57677541a8ef None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:31.302418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4935aa4-db7d-4f9b-a67c-57677541a8ef None admin] RBAC: Adding query filter params (name=network) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:31.302724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4935aa4-db7d-4f9b-a67c-57677541a8ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:31.304167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4935aa4-db7d-4f9b-a67c-57677541a8ef None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:31.310068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 386/782] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:31.315556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4935aa4-db7d-4f9b-a67c-57677541a8ef None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:31.328583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9c40b-7ac8-4a13-bd10-723898c7d089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:31.329211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9c40b-7ac8-4a13-bd10-723898c7d089 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:31.329575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9c40b-7ac8-4a13-bd10-723898c7d089 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:31.329849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9c40b-7ac8-4a13-bd10-723898c7d089 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:31.330501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9c40b-7ac8-4a13-bd10-723898c7d089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:31.330807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9c40b-7ac8-4a13-bd10-723898c7d089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:31.332179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9c40b-7ac8-4a13-bd10-723898c7d089 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:31.368006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 397/783] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:31.373913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9c40b-7ac8-4a13-bd10-723898c7d089 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:31.461766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4935aa4-db7d-4f9b-a67c-57677541a8ef None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:31.500156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115281ff-c3ab-4c73-87f3-6f5f8d932e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:31.501415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:31.502907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115281ff-c3ab-4c73-87f3-6f5f8d932e09 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:31.503424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115281ff-c3ab-4c73-87f3-6f5f8d932e09 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:31.503941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115281ff-c3ab-4c73-87f3-6f5f8d932e09 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:31.506014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115281ff-c3ab-4c73-87f3-6f5f8d932e09 None admin] RBAC: Adding query filter params (type=network) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:31.507979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115281ff-c3ab-4c73-87f3-6f5f8d932e09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:31.509617 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115281ff-c3ab-4c73-87f3-6f5f8d932e09 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:31.520934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 387/784] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:31.522504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115281ff-c3ab-4c73-87f3-6f5f8d932e09 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:31.526692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:31.585592 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9c40b-7ac8-4a13-bd10-723898c7d089 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:31.664859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0196f-6dba-4ee9-8e23-1e66c18c44a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:31.665920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0196f-6dba-4ee9-8e23-1e66c18c44a8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:31.668032 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0196f-6dba-4ee9-8e23-1e66c18c44a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:31.668522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0196f-6dba-4ee9-8e23-1e66c18c44a8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:31.686569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0196f-6dba-4ee9-8e23-1e66c18c44a8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:31.688047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0196f-6dba-4ee9-8e23-1e66c18c44a8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:31.698728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115281ff-c3ab-4c73-87f3-6f5f8d932e09 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-47e0196f-6dba-4ee9-8e23-1e66c18c44a8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:31.711037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:31.711984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:31.712920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:31.712920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:31.712920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:31.712920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:31.712920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:31.712920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:31.712920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:31.712920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:31.712920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:31.712920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:31.718041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 398/785] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:31.719454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0196f-6dba-4ee9-8e23-1e66c18c44a8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:31.746081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a2d31-1b72-4536-b08b-53fbe44b1a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:31.747048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a2d31-1b72-4536-b08b-53fbe44b1a9b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:31.747559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a2d31-1b72-4536-b08b-53fbe44b1a9b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:31.748073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a2d31-1b72-4536-b08b-53fbe44b1a9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:31.749061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a2d31-1b72-4536-b08b-53fbe44b1a9b None admin] RBAC: Adding query filter params (interface=public, service_id=82f16a51a0804d069bbc375799b80f71, region_id=RegionOne) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:31.749658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a2d31-1b72-4536-b08b-53fbe44b1a9b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:31.752229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a2d31-1b72-4536-b08b-53fbe44b1a9b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:31.779414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 388/786] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 3 14:39:31 2022] GET /identity/v3/endpoints?service_id=82f16a51a0804d069bbc375799b80f71&interface=public®ion_id=RegionOne => generated 194 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:31.785511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a2d31-1b72-4536-b08b-53fbe44b1a9b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:31.904009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:31.984933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a2d31-1b72-4536-b08b-53fbe44b1a9b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:32.048225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0196f-6dba-4ee9-8e23-1e66c18c44a8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:32.063720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e61bd-b972-477e-b0b1-bd022703e746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:32.064337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e61bd-b972-477e-b0b1-bd022703e746 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:32.064617 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e61bd-b972-477e-b0b1-bd022703e746 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:32.064948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e61bd-b972-477e-b0b1-bd022703e746 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:32.065585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e61bd-b972-477e-b0b1-bd022703e746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:32.067021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e61bd-b972-477e-b0b1-bd022703e746 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:32.073456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 389/787] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 3 14:39:31 2022] GET /identity/v3/services => generated 772 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:32.092000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10615e94-052b-4748-accd-a3a392fb6f4c None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:32.092339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10615e94-052b-4748-accd-a3a392fb6f4c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:32.092743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10615e94-052b-4748-accd-a3a392fb6f4c None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:32.094298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 390/788] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:32.101417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:32.106986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb14a-0168-4e57-9f2a-05f9ad0fbc76 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:32.107585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb14a-0168-4e57-9f2a-05f9ad0fbc76 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:32.108090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb14a-0168-4e57-9f2a-05f9ad0fbc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:32.117613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38208d8-2b25-4b8f-b23c-4554fafd6e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:32.117613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38208d8-2b25-4b8f-b23c-4554fafd6e3f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:32.117613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38208d8-2b25-4b8f-b23c-4554fafd6e3f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:32.117613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38208d8-2b25-4b8f-b23c-4554fafd6e3f None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:32.118042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38208d8-2b25-4b8f-b23c-4554fafd6e3f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:32.118042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38208d8-2b25-4b8f-b23c-4554fafd6e3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:32.118042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38208d8-2b25-4b8f-b23c-4554fafd6e3f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:32.135122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 399/789] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 3 14:39:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:32.143542 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38208d8-2b25-4b8f-b23c-4554fafd6e3f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:32.145271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-24dfb14a-0168-4e57-9f2a-05f9ad0fbc76 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:32.146782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-24dfb14a-0168-4e57-9f2a-05f9ad0fbc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:32.250962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38208d8-2b25-4b8f-b23c-4554fafd6e3f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:32.284299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 391/790] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:32.305603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922269-7b65-496c-9f58-540e273e8616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:32.306256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922269-7b65-496c-9f58-540e273e8616 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:32.306530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922269-7b65-496c-9f58-540e273e8616 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:32.306797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922269-7b65-496c-9f58-540e273e8616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:32.322968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33922269-7b65-496c-9f58-540e273e8616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:32.324672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33922269-7b65-496c-9f58-540e273e8616 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-33922269-7b65-496c-9f58-540e273e8616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:32.350557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:32.351574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:32.352474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:32.352474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:32.352474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:32.352474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 400/791] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:32.357550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfb14a-0168-4e57-9f2a-05f9ad0fbc76 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:32.425910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:32.465985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfb14a-0168-4e57-9f2a-05f9ad0fbc76 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:32.504091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb429b-eaf3-40ef-8a76-a5b3e0488b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:32.504903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb429b-eaf3-40ef-8a76-a5b3e0488b87 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:32.505092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb429b-eaf3-40ef-8a76-a5b3e0488b87 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:32.505253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb429b-eaf3-40ef-8a76-a5b3e0488b87 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:32.505897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb429b-eaf3-40ef-8a76-a5b3e0488b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:32.506297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb429b-eaf3-40ef-8a76-a5b3e0488b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:32.507670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb429b-eaf3-40ef-8a76-a5b3e0488b87 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:32.520987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 392/792] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:32.526734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922269-7b65-496c-9f58-540e273e8616 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:32.596041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:32.661780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb429b-eaf3-40ef-8a76-a5b3e0488b87 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:32.675393 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922269-7b65-496c-9f58-540e273e8616 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:32.714607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906fd4-280b-4f63-b694-d51b5c030818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:32.715248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906fd4-280b-4f63-b694-d51b5c030818 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:32.715524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906fd4-280b-4f63-b694-d51b5c030818 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:32.715954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906fd4-280b-4f63-b694-d51b5c030818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:32.723654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65906fd4-280b-4f63-b694-d51b5c030818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:32.725284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65906fd4-280b-4f63-b694-d51b5c030818 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-65906fd4-280b-4f63-b694-d51b5c030818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:32.737397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:32.738372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:32.739530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:32.739530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:32.739530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:32.739530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:32.739530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:32.739530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:32.739530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:32.739530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:32.739530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 401/793] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:32.745471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906fd4-280b-4f63-b694-d51b5c030818 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:32.850485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:32.852768 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906fd4-280b-4f63-b694-d51b5c030818 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:32.898236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb429b-eaf3-40ef-8a76-a5b3e0488b87 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:32.933967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:32.942077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1d94c-c1bb-4ee3-aacb-ae3dcdab5900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:32.942743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1d94c-c1bb-4ee3-aacb-ae3dcdab5900 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:32.943024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1d94c-c1bb-4ee3-aacb-ae3dcdab5900 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:32.943297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1d94c-c1bb-4ee3-aacb-ae3dcdab5900 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:32.952033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fad8b-19c8-4bd1-be41-715bee4e5172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:32.952647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fad8b-19c8-4bd1-be41-715bee4e5172 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:32.952923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fad8b-19c8-4bd1-be41-715bee4e5172 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:32.953198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fad8b-19c8-4bd1-be41-715bee4e5172 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:32.953916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fad8b-19c8-4bd1-be41-715bee4e5172 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:32.954216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fad8b-19c8-4bd1-be41-715bee4e5172 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:32.955646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fad8b-19c8-4bd1-be41-715bee4e5172 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:32.971699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 402/794] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:32.977565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1d94c-c1bb-4ee3-aacb-ae3dcdab5900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:32.978066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fad8b-19c8-4bd1-be41-715bee4e5172 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-8ee1d94c-c1bb-4ee3-aacb-ae3dcdab5900 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:32.983121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:32.984049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:32.984890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 393/795] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:32.992826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1d94c-c1bb-4ee3-aacb-ae3dcdab5900 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:33.104796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fad8b-19c8-4bd1-be41-715bee4e5172 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:33.149279 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1d94c-c1bb-4ee3-aacb-ae3dcdab5900 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:33.152619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2660579-7ba5-4f9d-a929-275d46fc6847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:33.153288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2660579-7ba5-4f9d-a929-275d46fc6847 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:33.153592 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2660579-7ba5-4f9d-a929-275d46fc6847 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:33.153860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2660579-7ba5-4f9d-a929-275d46fc6847 None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/6fbf23ae20ee4f6c979e256d2d4daa5f/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:33.155780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2660579-7ba5-4f9d-a929-275d46fc6847 None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=6fbf23ae20ee4f6c979e256d2d4daa5f, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:33.157771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2660579-7ba5-4f9d-a929-275d46fc6847 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:33.177765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 403/796] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:39:32 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/6fbf23ae20ee4f6c979e256d2d4daa5f/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:39:33.179687 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0fb61-1fc2-4873-913d-eff1ce6ab9c4 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:33.179970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0fb61-1fc2-4873-913d-eff1ce6ab9c4 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:33.180245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0fb61-1fc2-4873-913d-eff1ce6ab9c4 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:33.185743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 404/797] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:33.186973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5af144-a739-4723-9161-2b14022a4f75 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:33.187251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5af144-a739-4723-9161-2b14022a4f75 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:33.187519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5af144-a739-4723-9161-2b14022a4f75 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:33.190401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f072ace-4cf7-4e3f-ae60-8677cb9c5d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:33.191031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f072ace-4cf7-4e3f-ae60-8677cb9c5d33 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:33.191305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f072ace-4cf7-4e3f-ae60-8677cb9c5d33 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:33.191574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f072ace-4cf7-4e3f-ae60-8677cb9c5d33 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:33.192295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f072ace-4cf7-4e3f-ae60-8677cb9c5d33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:33.192574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f072ace-4cf7-4e3f-ae60-8677cb9c5d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:33.194700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f072ace-4cf7-4e3f-ae60-8677cb9c5d33 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:33.197974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 405/798] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:33.199427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3e322-6b24-4daf-8dd4-563c6e29a4b9 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:33.199771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3e322-6b24-4daf-8dd4-563c6e29a4b9 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:33.200043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3e322-6b24-4daf-8dd4-563c6e29a4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:33.209952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 394/799] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:33.211645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9fed-924a-4a8e-88ab-308cf8384b7a None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:33.212159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9fed-924a-4a8e-88ab-308cf8384b7a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:33.212635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9fed-924a-4a8e-88ab-308cf8384b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:33.223787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-4ab3e322-6b24-4daf-8dd4-563c6e29a4b9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:33.225068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4ab3e322-6b24-4daf-8dd4-563c6e29a4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:33.233094 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2c9a9fed-924a-4a8e-88ab-308cf8384b7a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:33.234698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2c9a9fed-924a-4a8e-88ab-308cf8384b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:33.243396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:33.284068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:33.320761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 395/800] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:33.320761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9fed-924a-4a8e-88ab-308cf8384b7a None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:33.322099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 406/801] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:33.428350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9fed-924a-4a8e-88ab-308cf8384b7a None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:33.464868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb2727-ee98-4b3f-8f0e-2c8d6b73bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:33.465614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb2727-ee98-4b3f-8f0e-2c8d6b73bdda None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:33.465900 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb2727-ee98-4b3f-8f0e-2c8d6b73bdda None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:33.466174 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb2727-ee98-4b3f-8f0e-2c8d6b73bdda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:33.476118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb2727-ee98-4b3f-8f0e-2c8d6b73bdda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:33.477596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb2727-ee98-4b3f-8f0e-2c8d6b73bdda None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-ffbb2727-ee98-4b3f-8f0e-2c8d6b73bdda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:33.492197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:33.493121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:33.494022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:33.494022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:33.494022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:33.494022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 396/802] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:33.498980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3e322-6b24-4daf-8dd4-563c6e29a4b9 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:33.628121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3e322-6b24-4daf-8dd4-563c6e29a4b9 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:33.660829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb2727-ee98-4b3f-8f0e-2c8d6b73bdda None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:33.672813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2b803-957b-44b8-a58f-2a4a2237d639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:33.673764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b803-957b-44b8-a58f-2a4a2237d639 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:33.674692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b803-957b-44b8-a58f-2a4a2237d639 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:33.675227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b803-957b-44b8-a58f-2a4a2237d639 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:33.676134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2b803-957b-44b8-a58f-2a4a2237d639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:33.676659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2b803-957b-44b8-a58f-2a4a2237d639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:33.678508 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2b803-957b-44b8-a58f-2a4a2237d639 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:33.692050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 407/803] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:33.698499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2b803-957b-44b8-a58f-2a4a2237d639 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:33.712293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:33.725862 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:33.835011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb2727-ee98-4b3f-8f0e-2c8d6b73bdda None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:33.850432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2b803-957b-44b8-a58f-2a4a2237d639 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:33.874575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10149c77-50fe-49b9-8ce8-820043bf38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:33.875540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10149c77-50fe-49b9-8ce8-820043bf38aa None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:33.876177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10149c77-50fe-49b9-8ce8-820043bf38aa None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:33.876640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10149c77-50fe-49b9-8ce8-820043bf38aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:33.888371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10149c77-50fe-49b9-8ce8-820043bf38aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:33.888794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574bad8-8a9a-4eda-8cc0-718827a14de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:33.889485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574bad8-8a9a-4eda-8cc0-718827a14de2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:33.889864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574bad8-8a9a-4eda-8cc0-718827a14de2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:33.890197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574bad8-8a9a-4eda-8cc0-718827a14de2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-10149c77-50fe-49b9-8ce8-820043bf38aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:33.894838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:33.895944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:33.897548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:33.900057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 397/804] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:33.900985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10149c77-50fe-49b9-8ce8-820043bf38aa None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:33.901696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574bad8-8a9a-4eda-8cc0-718827a14de2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:33.903305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574bad8-8a9a-4eda-8cc0-718827a14de2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-7574bad8-8a9a-4eda-8cc0-718827a14de2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:33.916649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:33.917803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:33.918727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:33.918727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:33.918727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:33.918727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:33.918727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:33.918727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:33.918727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:33.918727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:33.918727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:39:33.918727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:33.921153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 408/805] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 3 14:39:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:33.922124 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574bad8-8a9a-4eda-8cc0-718827a14de2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:33.994635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:34.002424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:34.019293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10149c77-50fe-49b9-8ce8-820043bf38aa None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:34.031936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574bad8-8a9a-4eda-8cc0-718827a14de2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:34.056200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9666b0-1ccf-447e-bbb9-c3a8cda5d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:34.057011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9666b0-1ccf-447e-bbb9-c3a8cda5d18f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.057361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9666b0-1ccf-447e-bbb9-c3a8cda5d18f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.057729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9666b0-1ccf-447e-bbb9-c3a8cda5d18f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.058395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9666b0-1ccf-447e-bbb9-c3a8cda5d18f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:34.059835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9666b0-1ccf-447e-bbb9-c3a8cda5d18f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:34.069033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761c124-b27b-4b37-802b-66f212efc204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:34.069910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761c124-b27b-4b37-802b-66f212efc204 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-ad9666b0-1ccf-447e-bbb9-c3a8cda5d18f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:34.070041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:34.070954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:34.072037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 398/806] 10.222.0.46 () {56 vars in 1129 bytes} [Mon Oct 3 14:39:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:34.073128 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9666b0-1ccf-447e-bbb9-c3a8cda5d18f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:34.073665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761c124-b27b-4b37-802b-66f212efc204 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.074140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761c124-b27b-4b37-802b-66f212efc204 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.075016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761c124-b27b-4b37-802b-66f212efc204 None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:34.075516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761c124-b27b-4b37-802b-66f212efc204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:34.077642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761c124-b27b-4b37-802b-66f212efc204 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:34.087631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 409/807] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:34.088818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761c124-b27b-4b37-802b-66f212efc204 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:34.180421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9666b0-1ccf-447e-bbb9-c3a8cda5d18f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:34.195997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761c124-b27b-4b37-802b-66f212efc204 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:34.215669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0947f65-3c6b-46c4-ba87-5afcb49d8871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:34.216276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0947f65-3c6b-46c4-ba87-5afcb49d8871 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.216604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0947f65-3c6b-46c4-ba87-5afcb49d8871 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.216883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0947f65-3c6b-46c4-ba87-5afcb49d8871 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.217644 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0947f65-3c6b-46c4-ba87-5afcb49d8871 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:34.217951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0947f65-3c6b-46c4-ba87-5afcb49d8871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:34.219466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0947f65-3c6b-46c4-ba87-5afcb49d8871 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:34.230635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:34.239249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a317cc-d770-4b89-a139-3525dfb6da3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:34.239912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a317cc-d770-4b89-a139-3525dfb6da3b None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.240195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a317cc-d770-4b89-a139-3525dfb6da3b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.240528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a317cc-d770-4b89-a139-3525dfb6da3b None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.241224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a317cc-d770-4b89-a139-3525dfb6da3b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:34.241567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a317cc-d770-4b89-a139-3525dfb6da3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:34.243224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a317cc-d770-4b89-a139-3525dfb6da3b None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:34.243336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 399/808] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 3 14:39:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:34.244171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0947f65-3c6b-46c4-ba87-5afcb49d8871 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:34.253066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 410/809] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 3 14:39:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:34.253986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a317cc-d770-4b89-a139-3525dfb6da3b None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:34.257703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:34.366188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a317cc-d770-4b89-a139-3525dfb6da3b None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:34.373668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0947f65-3c6b-46c4-ba87-5afcb49d8871 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:34.403263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf343a0-8a46-4d42-834e-b8111a16fe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:34.404264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf343a0-8a46-4d42-834e-b8111a16fe8b None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.404864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf343a0-8a46-4d42-834e-b8111a16fe8b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.405358 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf343a0-8a46-4d42-834e-b8111a16fe8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.477884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:34.489882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf343a0-8a46-4d42-834e-b8111a16fe8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:34.491561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf343a0-8a46-4d42-834e-b8111a16fe8b None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:34.537041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a7b3e-ad97-4bfc-9a0b-b0c48ba492a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:34.538439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a7b3e-ad97-4bfc-9a0b-b0c48ba492a8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.538720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a7b3e-ad97-4bfc-9a0b-b0c48ba492a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.538994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a7b3e-ad97-4bfc-9a0b-b0c48ba492a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.550777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a7b3e-ad97-4bfc-9a0b-b0c48ba492a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:34.552249 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a7b3e-ad97-4bfc-9a0b-b0c48ba492a8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-d56a7b3e-ad97-4bfc-9a0b-b0c48ba492a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:34.570503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:34.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:34.572447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:34.572447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:34.572447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:34.576576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 400/810] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:34.577600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a7b3e-ad97-4bfc-9a0b-b0c48ba492a8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:34.605422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 411/811] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:39:34.607146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf343a0-8a46-4d42-834e-b8111a16fe8b None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:34.654280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:34.704267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a7b3e-ad97-4bfc-9a0b-b0c48ba492a8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:34.718140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:34.727757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf343a0-8a46-4d42-834e-b8111a16fe8b None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:34.739838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7855dc5-ca7a-46c2-be52-4c465c4e03b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:34.740973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7855dc5-ca7a-46c2-be52-4c465c4e03b3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.741257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7855dc5-ca7a-46c2-be52-4c465c4e03b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.741547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7855dc5-ca7a-46c2-be52-4c465c4e03b3 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.742183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7855dc5-ca7a-46c2-be52-4c465c4e03b3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:34.742483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7855dc5-ca7a-46c2-be52-4c465c4e03b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:34.743845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7855dc5-ca7a-46c2-be52-4c465c4e03b3 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:34.748854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 401/812] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 3 14:39:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:34.754037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7855dc5-ca7a-46c2-be52-4c465c4e03b3 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:34.765996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49268937-b24a-4e55-9ad7-5186fbbf67e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:34.766912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49268937-b24a-4e55-9ad7-5186fbbf67e7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.767193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49268937-b24a-4e55-9ad7-5186fbbf67e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.767614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49268937-b24a-4e55-9ad7-5186fbbf67e7 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.768260 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49268937-b24a-4e55-9ad7-5186fbbf67e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:34.768560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49268937-b24a-4e55-9ad7-5186fbbf67e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:34.770049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49268937-b24a-4e55-9ad7-5186fbbf67e7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:34.785960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 412/813] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:34.785960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd7dea-99da-49b8-ad5c-e3dc3144f5f6 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.785960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd7dea-99da-49b8-ad5c-e3dc3144f5f6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.785960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd7dea-99da-49b8-ad5c-e3dc3144f5f6 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.785960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 413/814] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:34.785960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd7dea-99da-49b8-ad5c-e3dc3144f5f6 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:34.864567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7855dc5-ca7a-46c2-be52-4c465c4e03b3 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:34.887076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd7dea-99da-49b8-ad5c-e3dc3144f5f6 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:34.888331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:34.900325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47445b8a-3378-4197-b9e0-2db56063fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:34.901007 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47445b8a-3378-4197-b9e0-2db56063fec7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.901289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47445b8a-3378-4197-b9e0-2db56063fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.901591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47445b8a-3378-4197-b9e0-2db56063fec7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.913205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47445b8a-3378-4197-b9e0-2db56063fec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:34.914679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47445b8a-3378-4197-b9e0-2db56063fec7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:34.924196 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f57add-44de-452b-93e3-6f007d9511e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:34.925029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57add-44de-452b-93e3-6f007d9511e2 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.925313 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57add-44de-452b-93e3-6f007d9511e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.925637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57add-44de-452b-93e3-6f007d9511e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.926284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f57add-44de-452b-93e3-6f007d9511e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:34.927625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f57add-44de-452b-93e3-6f007d9511e2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-47445b8a-3378-4197-b9e0-2db56063fec7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:34.931348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:34.932256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:34.933335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:34.933335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:34.933335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:34.939586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 402/815] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:34.939586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47445b8a-3378-4197-b9e0-2db56063fec7 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:34.965777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 414/816] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:39:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:39:34.967391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996174db-ff65-4b41-8b4f-a6bb7e6f4fea None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:34.967678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996174db-ff65-4b41-8b4f-a6bb7e6f4fea None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:34.967950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996174db-ff65-4b41-8b4f-a6bb7e6f4fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:34.995294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-996174db-ff65-4b41-8b4f-a6bb7e6f4fea None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:34.996559 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-996174db-ff65-4b41-8b4f-a6bb7e6f4fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:35.007833 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:35.075412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47445b8a-3378-4197-b9e0-2db56063fec7 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:35.121394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 415/817] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:35.123223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996174db-ff65-4b41-8b4f-a6bb7e6f4fea None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:35.127483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ef9ad-f315-497c-bbc5-e3406e23beb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:35.128186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ef9ad-f315-497c-bbc5-e3406e23beb0 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.128464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ef9ad-f315-497c-bbc5-e3406e23beb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.128742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ef9ad-f315-497c-bbc5-e3406e23beb0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.137621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ef9ad-f315-497c-bbc5-e3406e23beb0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:35.139200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ef9ad-f315-497c-bbc5-e3406e23beb0 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-9e6ef9ad-f315-497c-bbc5-e3406e23beb0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:35.151854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:35.152854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:35.153826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 403/818] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 3 14:39:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:35.159223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ef9ad-f315-497c-bbc5-e3406e23beb0 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:35.202845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:35.269491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996174db-ff65-4b41-8b4f-a6bb7e6f4fea None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:35.297026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ef9ad-f315-497c-bbc5-e3406e23beb0 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:35.320927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df88d9d-7605-4741-ad53-26ede8ecdf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:35.322132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88d9d-7605-4741-ad53-26ede8ecdf7a None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.323018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88d9d-7605-4741-ad53-26ede8ecdf7a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.323487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88d9d-7605-4741-ad53-26ede8ecdf7a None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.324577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df88d9d-7605-4741-ad53-26ede8ecdf7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:35.325272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df88d9d-7605-4741-ad53-26ede8ecdf7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:35.327477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df88d9d-7605-4741-ad53-26ede8ecdf7a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:35.335708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f095d-4a5d-45c7-9585-6452a24e0088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:35.336450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f095d-4a5d-45c7-9585-6452a24e0088 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.336670 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:35.336919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f095d-4a5d-45c7-9585-6452a24e0088 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.337779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f095d-4a5d-45c7-9585-6452a24e0088 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.338534 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f095d-4a5d-45c7-9585-6452a24e0088 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:35.339979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f095d-4a5d-45c7-9585-6452a24e0088 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:35.341528 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f095d-4a5d-45c7-9585-6452a24e0088 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:35.343728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 416/819] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:35.350522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df88d9d-7605-4741-ad53-26ede8ecdf7a None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:35.363062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 404/820] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 3 14:39:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:35.371244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f095d-4a5d-45c7-9585-6452a24e0088 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:35.469541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:35.479128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df88d9d-7605-4741-ad53-26ede8ecdf7a None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:35.480447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f095d-4a5d-45c7-9585-6452a24e0088 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:35.516013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af11f0-7c01-4fff-9f3b-ba94d68424ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:35.516637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af11f0-7c01-4fff-9f3b-ba94d68424ea None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.516919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af11f0-7c01-4fff-9f3b-ba94d68424ea None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.517186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af11f0-7c01-4fff-9f3b-ba94d68424ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.519266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c239-e1d2-4275-8719-dae1a7c0d05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:35.520507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c239-e1d2-4275-8719-dae1a7c0d05f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.520940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c239-e1d2-4275-8719-dae1a7c0d05f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.521363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c239-e1d2-4275-8719-dae1a7c0d05f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.525011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af11f0-7c01-4fff-9f3b-ba94d68424ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:35.526515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af11f0-7c01-4fff-9f3b-ba94d68424ea None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:35.533375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670c239-e1d2-4275-8719-dae1a7c0d05f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:35.535006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670c239-e1d2-4275-8719-dae1a7c0d05f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-26af11f0-7c01-4fff-9f3b-ba94d68424ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:35.538652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:35.539700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:35.540798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:35.540798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:35.540798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:35.540798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:35.540798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:35.540798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:35.540798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:35.540798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:35.540798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 417/821] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:35.541370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af11f0-7c01-4fff-9f3b-ba94d68424ea None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-a670c239-e1d2-4275-8719-dae1a7c0d05f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:35.560616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:35.561873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:35.562946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:35.562946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:35.562946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:35.569529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 405/822] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:35.569529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c239-e1d2-4275-8719-dae1a7c0d05f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:35.622259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:35.668576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af11f0-7c01-4fff-9f3b-ba94d68424ea None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:35.698874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c239-e1d2-4275-8719-dae1a7c0d05f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:35.705362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78bc21-be4b-4103-a0e5-273d40ff4bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:35.706123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78bc21-be4b-4103-a0e5-273d40ff4bb0 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.706403 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78bc21-be4b-4103-a0e5-273d40ff4bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.706672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78bc21-be4b-4103-a0e5-273d40ff4bb0 None admin] PATH_INFO: `/v3/services/network` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.707257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78bc21-be4b-4103-a0e5-273d40ff4bb0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:35.708864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78bc21-be4b-4103-a0e5-273d40ff4bb0 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-6a78bc21-be4b-4103-a0e5-273d40ff4bb0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:35.717930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:35.718926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:35.719823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 418/823] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:35 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:35.720634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78bc21-be4b-4103-a0e5-273d40ff4bb0 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:35.742142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590bdd9-041a-478a-a311-0f02ec40c224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:35.742810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590bdd9-041a-478a-a311-0f02ec40c224 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.743031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590bdd9-041a-478a-a311-0f02ec40c224 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.743305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590bdd9-041a-478a-a311-0f02ec40c224 None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.744061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590bdd9-041a-478a-a311-0f02ec40c224 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:35.744372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590bdd9-041a-478a-a311-0f02ec40c224 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:35.745828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590bdd9-041a-478a-a311-0f02ec40c224 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:35.758655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 406/824] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:35.759854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d316939-fc2d-456e-afa4-068a1e00fecd None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.760140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d316939-fc2d-456e-afa4-068a1e00fecd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.760412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d316939-fc2d-456e-afa4-068a1e00fecd None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.761627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 407/825] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:35.765893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d316939-fc2d-456e-afa4-068a1e00fecd None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:35.768076 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:35.828949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78bc21-be4b-4103-a0e5-273d40ff4bb0 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:35.854843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:35.871295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3c78-7d8c-4353-bb68-573bf7baa81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:35.871896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3c78-7d8c-4353-bb68-573bf7baa81e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.872164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3c78-7d8c-4353-bb68-573bf7baa81e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.872428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3c78-7d8c-4353-bb68-573bf7baa81e None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.873060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd3c78-7d8c-4353-bb68-573bf7baa81e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:35.873356 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd3c78-7d8c-4353-bb68-573bf7baa81e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:35.873909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d316939-fc2d-456e-afa4-068a1e00fecd None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:35.874748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd3c78-7d8c-4353-bb68-573bf7baa81e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:35.888114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 419/826] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:35.888931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3c78-7d8c-4353-bb68-573bf7baa81e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:35.914626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eadb1-6b38-4db4-903e-7dce864b5c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:35.915552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eadb1-6b38-4db4-903e-7dce864b5c37 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.915835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eadb1-6b38-4db4-903e-7dce864b5c37 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.916109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eadb1-6b38-4db4-903e-7dce864b5c37 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.916735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eadb1-6b38-4db4-903e-7dce864b5c37 None admin] RBAC: Adding query filter params (name=network) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:35.917036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eadb1-6b38-4db4-903e-7dce864b5c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:35.918434 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eadb1-6b38-4db4-903e-7dce864b5c37 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:35.923209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 408/827] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:35.928769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599b03-1a4b-4a3a-af6e-f5c54c34422e None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:35.929057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599b03-1a4b-4a3a-af6e-f5c54c34422e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:35.929333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599b03-1a4b-4a3a-af6e-f5c54c34422e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:35.948083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d8599b03-1a4b-4a3a-af6e-f5c54c34422e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:35.949652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d8599b03-1a4b-4a3a-af6e-f5c54c34422e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:35.993196 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3c78-7d8c-4353-bb68-573bf7baa81e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:36.017004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:36.026959 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 409/828] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:36.027927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8599b03-1a4b-4a3a-af6e-f5c54c34422e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:36.041151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38f7bd-e232-4dd6-b707-74455edac918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:36.042071 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f7bd-e232-4dd6-b707-74455edac918 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:36.042556 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f7bd-e232-4dd6-b707-74455edac918 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:36.043038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f7bd-e232-4dd6-b707-74455edac918 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:36.044108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38f7bd-e232-4dd6-b707-74455edac918 None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=1a4a95ae5ee84f259ffcd7528b0fa505) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:36.044498 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38f7bd-e232-4dd6-b707-74455edac918 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:36.046051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38f7bd-e232-4dd6-b707-74455edac918 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:36.053621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 420/829] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:35 2022] GET /identity/v3/role_assignments?user.id=1a4a95ae5ee84f259ffcd7528b0fa505&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:36.054725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38f7bd-e232-4dd6-b707-74455edac918 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:36.117099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:36.201269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8599b03-1a4b-4a3a-af6e-f5c54c34422e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:36.235926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38f7bd-e232-4dd6-b707-74455edac918 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:36.281473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfed95-dd8e-4a85-bb54-3b03e829d0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:36.283372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfed95-dd8e-4a85-bb54-3b03e829d0ba None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:36.283667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfed95-dd8e-4a85-bb54-3b03e829d0ba None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:36.283939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfed95-dd8e-4a85-bb54-3b03e829d0ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:36.292706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc03f9f-74c9-4cd0-a4fe-fc31a5e482af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:36.293444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc03f9f-74c9-4cd0-a4fe-fc31a5e482af None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:36.293835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc03f9f-74c9-4cd0-a4fe-fc31a5e482af None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:36.294218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc03f9f-74c9-4cd0-a4fe-fc31a5e482af None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:36.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc03f9f-74c9-4cd0-a4fe-fc31a5e482af None admin] RBAC: Adding query filter params (type=network) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:36.295567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc03f9f-74c9-4cd0-a4fe-fc31a5e482af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:36.296525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfed95-dd8e-4a85-bb54-3b03e829d0ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:36.297191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc03f9f-74c9-4cd0-a4fe-fc31a5e482af None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:36.298141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfed95-dd8e-4a85-bb54-3b03e829d0ba None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:36.303900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 421/830] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:36.309714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc03f9f-74c9-4cd0-a4fe-fc31a5e482af None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-75dfed95-dd8e-4a85-bb54-3b03e829d0ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:36.313718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:36.314648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:36.315512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:36.315512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:36.315512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:36.315512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:36.315512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:36.315512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:36.315512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:36.315512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:36.315512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 410/831] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:36.320620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfed95-dd8e-4a85-bb54-3b03e829d0ba None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:36.395215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:36.439016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc03f9f-74c9-4cd0-a4fe-fc31a5e482af None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:36.450343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:36.453561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfed95-dd8e-4a85-bb54-3b03e829d0ba None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:36.475651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e4072-7520-4040-8d86-18154a31222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:36.476259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e4072-7520-4040-8d86-18154a31222b None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:36.476534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e4072-7520-4040-8d86-18154a31222b None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:36.476802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e4072-7520-4040-8d86-18154a31222b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:36.477465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e4072-7520-4040-8d86-18154a31222b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:36.478867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e4072-7520-4040-8d86-18154a31222b None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:36.496177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5f639-3233-4750-b812-1eb1ea1eac20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:36.497215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5f639-3233-4750-b812-1eb1ea1eac20 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:36.497722 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5f639-3233-4750-b812-1eb1ea1eac20 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:36.498187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5f639-3233-4750-b812-1eb1ea1eac20 None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:36.499072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5f639-3233-4750-b812-1eb1ea1eac20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:36.499588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5f639-3233-4750-b812-1eb1ea1eac20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:36.501283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5f639-3233-4750-b812-1eb1ea1eac20 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:36.501994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 422/832] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:39:36 2022] POST /identity/v3/endpoints => generated 337 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:39:36.509701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e4072-7520-4040-8d86-18154a31222b None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:36.527943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 411/833] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:36.533855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5f639-3233-4750-b812-1eb1ea1eac20 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:36.656484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e4072-7520-4040-8d86-18154a31222b None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:36.687295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5f639-3233-4750-b812-1eb1ea1eac20 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:36.708052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be8b78-295a-496a-ba22-76529c10a5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:36.708688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be8b78-295a-496a-ba22-76529c10a5f3 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:36.708991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be8b78-295a-496a-ba22-76529c10a5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:36.709260 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be8b78-295a-496a-ba22-76529c10a5f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:36.720049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be8b78-295a-496a-ba22-76529c10a5f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:36.723831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-f5be8b78-295a-496a-ba22-76529c10a5f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:36.727020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:36.727937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:36.728752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 423/834] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:36.739478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be8b78-295a-496a-ba22-76529c10a5f3 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:36.760055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4d872-4b79-45cc-a3db-958353bb5753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:36.761480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4d872-4b79-45cc-a3db-958353bb5753 None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:36.761982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4d872-4b79-45cc-a3db-958353bb5753 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:36.762465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4d872-4b79-45cc-a3db-958353bb5753 None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/fd0db9d2a753490cb8473229a8f6fe15/roles/f5822988729a45539e07519e59cf7bc4` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:36.767445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:36.780314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4d872-4b79-45cc-a3db-958353bb5753 None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=fd0db9d2a753490cb8473229a8f6fe15, role_id=f5822988729a45539e07519e59cf7bc4)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:36.782935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4d872-4b79-45cc-a3db-958353bb5753 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:36.798184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 412/835] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:39:36 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/fd0db9d2a753490cb8473229a8f6fe15/roles/f5822988729a45539e07519e59cf7bc4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:39:36.859558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be8b78-295a-496a-ba22-76529c10a5f3 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:36.905529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8484892-4266-4d64-bdf4-6a929c5c7005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:36.906143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8484892-4266-4d64-bdf4-6a929c5c7005 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:36.906421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8484892-4266-4d64-bdf4-6a929c5c7005 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:36.906733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8484892-4266-4d64-bdf4-6a929c5c7005 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:36.907421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8484892-4266-4d64-bdf4-6a929c5c7005 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:36.907719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8484892-4266-4d64-bdf4-6a929c5c7005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:36.909183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8484892-4266-4d64-bdf4-6a929c5c7005 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:36.919188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 424/836] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:36.924976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4d872-4b79-45cc-a3db-958353bb5753 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:37.042326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4d872-4b79-45cc-a3db-958353bb5753 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:37.080957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9eb5aa-a78b-4c53-8d5c-14cf6fbdbd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:37.081523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9eb5aa-a78b-4c53-8d5c-14cf6fbdbd8a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:37.081818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9eb5aa-a78b-4c53-8d5c-14cf6fbdbd8a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:37.082085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9eb5aa-a78b-4c53-8d5c-14cf6fbdbd8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:37.093063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9eb5aa-a78b-4c53-8d5c-14cf6fbdbd8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:37.094556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9eb5aa-a78b-4c53-8d5c-14cf6fbdbd8a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-2c9eb5aa-a78b-4c53-8d5c-14cf6fbdbd8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:37.110687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:37.111673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:37.112556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:37.112556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:37.112556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:37.112556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 413/837] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:37.120073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8484892-4266-4d64-bdf4-6a929c5c7005 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:37.207793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:37.225075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8484892-4266-4d64-bdf4-6a929c5c7005 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:37.259274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbd9e4-8e5a-4267-89a8-a9150c0a2ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:37.259879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbd9e4-8e5a-4267-89a8-a9150c0a2ba5 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:37.260154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbd9e4-8e5a-4267-89a8-a9150c0a2ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:37.260418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbd9e4-8e5a-4267-89a8-a9150c0a2ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:37.261051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbd9e4-8e5a-4267-89a8-a9150c0a2ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:37.261346 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbd9e4-8e5a-4267-89a8-a9150c0a2ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:37.262724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbd9e4-8e5a-4267-89a8-a9150c0a2ba5 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:37.275097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 425/838] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:37.280493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9eb5aa-a78b-4c53-8d5c-14cf6fbdbd8a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:37.309811 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:37.403816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9eb5aa-a78b-4c53-8d5c-14cf6fbdbd8a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:37.437022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27808afc-e80d-4664-8291-c01e885b2b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:37.437682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808afc-e80d-4664-8291-c01e885b2b80 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:37.437958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808afc-e80d-4664-8291-c01e885b2b80 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:37.438225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808afc-e80d-4664-8291-c01e885b2b80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:37.446148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27808afc-e80d-4664-8291-c01e885b2b80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:37.447582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27808afc-e80d-4664-8291-c01e885b2b80 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-27808afc-e80d-4664-8291-c01e885b2b80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:37.458540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:37.459541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:37.461054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 414/839] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:39:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:37.465858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbd9e4-8e5a-4267-89a8-a9150c0a2ba5 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:37.486280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a95ce-537e-4a74-b96d-19283b4f9d0f None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:37.486873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a95ce-537e-4a74-b96d-19283b4f9d0f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:37.487375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a95ce-537e-4a74-b96d-19283b4f9d0f None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:37.489425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 415/840] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:37.494631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c9f63-1bbb-45d1-aafe-ade0c5366fe6 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:37.495136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c9f63-1bbb-45d1-aafe-ade0c5366fe6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:37.495651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c9f63-1bbb-45d1-aafe-ade0c5366fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:37.513989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-195c9f63-1bbb-45d1-aafe-ade0c5366fe6 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:37.515470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-195c9f63-1bbb-45d1-aafe-ade0c5366fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:37.574740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbd9e4-8e5a-4267-89a8-a9150c0a2ba5 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:37.598511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 416/841] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:37.612748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f819b6-54c5-4441-8f22-011a73e8a088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:37.613357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f819b6-54c5-4441-8f22-011a73e8a088 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:37.613917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f819b6-54c5-4441-8f22-011a73e8a088 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:37.614188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f819b6-54c5-4441-8f22-011a73e8a088 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:37.614911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f819b6-54c5-4441-8f22-011a73e8a088 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:37.615727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f819b6-54c5-4441-8f22-011a73e8a088 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:37.617144 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f819b6-54c5-4441-8f22-011a73e8a088 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:37.635122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 426/842] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 3 14:39:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:37.640313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c9f63-1bbb-45d1-aafe-ade0c5366fe6 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:37.651256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:37.746453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c9f63-1bbb-45d1-aafe-ade0c5366fe6 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:37.781251 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f249465-6fc6-4f34-b89c-3f2c588754a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:37.781933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f249465-6fc6-4f34-b89c-3f2c588754a8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:37.782200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f249465-6fc6-4f34-b89c-3f2c588754a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:37.782606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f249465-6fc6-4f34-b89c-3f2c588754a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:37.853213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f249465-6fc6-4f34-b89c-3f2c588754a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:37.854708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f249465-6fc6-4f34-b89c-3f2c588754a8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:37.864451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:37.871921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f819b6-54c5-4441-8f22-011a73e8a088 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:37.875110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:37.954672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 417/843] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:39:37.959791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f249465-6fc6-4f34-b89c-3f2c588754a8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:37.983216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f819b6-54c5-4441-8f22-011a73e8a088 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:38.024003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c107627-fe39-4d1c-9491-68feab9e31a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:38.024923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c107627-fe39-4d1c-9491-68feab9e31a7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:38.025671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c107627-fe39-4d1c-9491-68feab9e31a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:38.026168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c107627-fe39-4d1c-9491-68feab9e31a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:38.027507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c107627-fe39-4d1c-9491-68feab9e31a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:38.029904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c107627-fe39-4d1c-9491-68feab9e31a7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-7c107627-fe39-4d1c-9491-68feab9e31a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:38.041303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:38.042307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 14:39:38.043110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:38.045819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 427/844] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:39:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:38.051392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c107627-fe39-4d1c-9491-68feab9e31a7 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:38.075739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f249465-6fc6-4f34-b89c-3f2c588754a8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:38.086555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:38.112126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b12520-1b2b-45d7-b88b-01187dedf05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:38.112758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b12520-1b2b-45d7-b88b-01187dedf05a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:38.113033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b12520-1b2b-45d7-b88b-01187dedf05a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:38.113303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b12520-1b2b-45d7-b88b-01187dedf05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:38.124269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b12520-1b2b-45d7-b88b-01187dedf05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:38.125570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b12520-1b2b-45d7-b88b-01187dedf05a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:38.131811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-61b12520-1b2b-45d7-b88b-01187dedf05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:38.141176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:38.142168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:38.143382 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:38.143382 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:38.143382 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:38.143382 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 418/845] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:38.150570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b12520-1b2b-45d7-b88b-01187dedf05a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:38.174792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c107627-fe39-4d1c-9491-68feab9e31a7 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:38.220403 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d3f48-5b43-4b8b-a00e-2975ec53eabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:38.221359 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d3f48-5b43-4b8b-a00e-2975ec53eabe None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:38.221729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d3f48-5b43-4b8b-a00e-2975ec53eabe None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:38.222069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d3f48-5b43-4b8b-a00e-2975ec53eabe None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:38.222904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d3f48-5b43-4b8b-a00e-2975ec53eabe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:38.223324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d3f48-5b43-4b8b-a00e-2975ec53eabe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:38.225978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d3f48-5b43-4b8b-a00e-2975ec53eabe None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:38.232767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 428/846] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:39:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:38.238474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d3f48-5b43-4b8b-a00e-2975ec53eabe None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:38.274808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b12520-1b2b-45d7-b88b-01187dedf05a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:38.317442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c80dc-fe2a-49e4-a44e-40807da04924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:38.318209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c80dc-fe2a-49e4-a44e-40807da04924 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:38.318640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c80dc-fe2a-49e4-a44e-40807da04924 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:38.319484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c80dc-fe2a-49e4-a44e-40807da04924 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:38.320250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c80dc-fe2a-49e4-a44e-40807da04924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:38.320654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c80dc-fe2a-49e4-a44e-40807da04924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:38.322593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c80dc-fe2a-49e4-a44e-40807da04924 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:38.341650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 419/847] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:38.343468 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c80dc-fe2a-49e4-a44e-40807da04924 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:38.351677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d3f48-5b43-4b8b-a00e-2975ec53eabe None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:38.367921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:38.390719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:38.400271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ef06-9555-46ef-955b-9efb78c640e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:38.401376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ef06-9555-46ef-955b-9efb78c640e1 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:38.401874 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ef06-9555-46ef-955b-9efb78c640e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:38.402573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ef06-9555-46ef-955b-9efb78c640e1 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:38.403807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ef06-9555-46ef-955b-9efb78c640e1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:38.404316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ef06-9555-46ef-955b-9efb78c640e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:38.406456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ef06-9555-46ef-955b-9efb78c640e1 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:38.417979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 429/848] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:39:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:38.467405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c80dc-fe2a-49e4-a44e-40807da04924 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:38.508144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6a69a-e2db-42e7-8b0c-20fe406b30f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:38.508799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6a69a-e2db-42e7-8b0c-20fe406b30f5 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:38.509080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6a69a-e2db-42e7-8b0c-20fe406b30f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:38.509350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6a69a-e2db-42e7-8b0c-20fe406b30f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:38.517179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6a69a-e2db-42e7-8b0c-20fe406b30f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:38.518701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6a69a-e2db-42e7-8b0c-20fe406b30f5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-e6b6a69a-e2db-42e7-8b0c-20fe406b30f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:38.531918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:38.532831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:38.533730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:38.533730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:38.533730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:38.533730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:38.533730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:38.533730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:38.533730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:38.533730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:38.533730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 420/849] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:38.541164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ef06-9555-46ef-955b-9efb78c640e1 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:38.660284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ef06-9555-46ef-955b-9efb78c640e1 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:38.700147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1f973-e286-4e81-9295-064c27ad9209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:38.700755 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1f973-e286-4e81-9295-064c27ad9209 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:38.701035 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1f973-e286-4e81-9295-064c27ad9209 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:38.701308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1f973-e286-4e81-9295-064c27ad9209 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:38.702039 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1f973-e286-4e81-9295-064c27ad9209 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:38.702349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1f973-e286-4e81-9295-064c27ad9209 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:38.703778 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1f973-e286-4e81-9295-064c27ad9209 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:38.714507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a164244-278d-4c0a-98cc-9aa91ebd1bb0 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:38.714773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a164244-278d-4c0a-98cc-9aa91ebd1bb0 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:38.715060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a164244-278d-4c0a-98cc-9aa91ebd1bb0 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:38.716752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 421/850] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:38.717584 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 430/851] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:38.721767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac13e3-8fbd-4521-94b5-ef1685cdb51c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:38.722055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac13e3-8fbd-4521-94b5-ef1685cdb51c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:38.722571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac13e3-8fbd-4521-94b5-ef1685cdb51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:38.723511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1f973-e286-4e81-9295-064c27ad9209 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:38.740512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-8aac13e3-8fbd-4521-94b5-ef1685cdb51c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:38.741732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8aac13e3-8fbd-4521-94b5-ef1685cdb51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:38.786683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:38.820458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 422/852] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:38.839445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1f973-e286-4e81-9295-064c27ad9209 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:38.876959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc563a30-807c-4556-8502-447567a87b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:38.877940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc563a30-807c-4556-8502-447567a87b9c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:38.878404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc563a30-807c-4556-8502-447567a87b9c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:38.878868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc563a30-807c-4556-8502-447567a87b9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:38.879786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc563a30-807c-4556-8502-447567a87b9c None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=6fbf23ae20ee4f6c979e256d2d4daa5f) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:38.880292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc563a30-807c-4556-8502-447567a87b9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:38.882037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc563a30-807c-4556-8502-447567a87b9c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:38.888601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 431/853] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:38 2022] GET /identity/v3/role_assignments?user.id=6fbf23ae20ee4f6c979e256d2d4daa5f&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:39.091402 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac13e3-8fbd-4521-94b5-ef1685cdb51c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:39.103730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:39.113252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2a49-e576-4f29-817a-b56f5c74d4b4 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:39.113969 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2a49-e576-4f29-817a-b56f5c74d4b4 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:39.114260 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2a49-e576-4f29-817a-b56f5c74d4b4 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:39.117019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 432/854] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:39.121924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4917d5-c9ad-4db2-8d4c-2999a0d8d678 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:39.122494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4917d5-c9ad-4db2-8d4c-2999a0d8d678 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:39.122771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4917d5-c9ad-4db2-8d4c-2999a0d8d678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:39.151181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-4a4917d5-c9ad-4db2-8d4c-2999a0d8d678 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:39.152373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4a4917d5-c9ad-4db2-8d4c-2999a0d8d678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:39.217998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac13e3-8fbd-4521-94b5-ef1685cdb51c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:39.247828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 433/855] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:39.269706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431096c-d28d-4373-b03e-acf61eba66e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:39.270583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431096c-d28d-4373-b03e-acf61eba66e2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:39.270938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431096c-d28d-4373-b03e-acf61eba66e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:39.271213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431096c-d28d-4373-b03e-acf61eba66e2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:39.272350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431096c-d28d-4373-b03e-acf61eba66e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:39.273744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431096c-d28d-4373-b03e-acf61eba66e2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-b431096c-d28d-4373-b03e-acf61eba66e2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:39.295038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:39.295928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:39.297199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 423/856] 10.222.0.46 () {56 vars in 1132 bytes} [Mon Oct 3 14:39:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:39.302320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4917d5-c9ad-4db2-8d4c-2999a0d8d678 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:39.363017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:39.418832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4917d5-c9ad-4db2-8d4c-2999a0d8d678 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:39.458244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ddb84-a001-4c18-8728-5cd668c31a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:39.459931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ddb84-a001-4c18-8728-5cd668c31a05 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:39.460430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ddb84-a001-4c18-8728-5cd668c31a05 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:39.460937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ddb84-a001-4c18-8728-5cd668c31a05 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:39.461829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ddb84-a001-4c18-8728-5cd668c31a05 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:39.462326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ddb84-a001-4c18-8728-5cd668c31a05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:39.463904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ddb84-a001-4c18-8728-5cd668c31a05 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:39.469530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 434/857] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:39.476001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431096c-d28d-4373-b03e-acf61eba66e2 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:39.562924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ddb84-a001-4c18-8728-5cd668c31a05 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:39.576108 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:39.604510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431096c-d28d-4373-b03e-acf61eba66e2 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:39.657942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f6f96-22e2-4686-ba7f-6011d7fec525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:39.658571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f6f96-22e2-4686-ba7f-6011d7fec525 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:39.658851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f6f96-22e2-4686-ba7f-6011d7fec525 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:39.659121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f6f96-22e2-4686-ba7f-6011d7fec525 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:39.659899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f6f96-22e2-4686-ba7f-6011d7fec525 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:39.660184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f6f96-22e2-4686-ba7f-6011d7fec525 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:39.661543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f6f96-22e2-4686-ba7f-6011d7fec525 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:39.666888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 424/858] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:39.706484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ddb84-a001-4c18-8728-5cd668c31a05 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:39.722247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:39.748457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd977be4-48f9-4816-a516-8776479b3a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:39.749489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977be4-48f9-4816-a516-8776479b3a37 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:39.749950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977be4-48f9-4816-a516-8776479b3a37 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:39.750363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977be4-48f9-4816-a516-8776479b3a37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:39.763344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd977be4-48f9-4816-a516-8776479b3a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-fd977be4-48f9-4816-a516-8776479b3a37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:39.769161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:39.770379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:39:39.771515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:39.774023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 435/859] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:39:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:39.780235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f6f96-22e2-4686-ba7f-6011d7fec525 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:39.892442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f6f96-22e2-4686-ba7f-6011d7fec525 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:39.927421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f941350-fdab-474f-bf8c-9552bc73749a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:39.928902 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f941350-fdab-474f-bf8c-9552bc73749a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:39.929375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f941350-fdab-474f-bf8c-9552bc73749a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:39.929836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f941350-fdab-474f-bf8c-9552bc73749a None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:39.930679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f941350-fdab-474f-bf8c-9552bc73749a None admin] RBAC: Adding query filter params (name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:39.931158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f941350-fdab-474f-bf8c-9552bc73749a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:39.932717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f941350-fdab-474f-bf8c-9552bc73749a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:39.938197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:39.943051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 425/860] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:39:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:39.949442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd977be4-48f9-4816-a516-8776479b3a37 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:40.058860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd977be4-48f9-4816-a516-8776479b3a37 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:40.096747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db66b4-884d-4ccd-9950-67c1da3f787e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:40.097378 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db66b4-884d-4ccd-9950-67c1da3f787e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:40.097784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db66b4-884d-4ccd-9950-67c1da3f787e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:40.098171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db66b4-884d-4ccd-9950-67c1da3f787e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:40.109561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db66b4-884d-4ccd-9950-67c1da3f787e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:40.111013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db66b4-884d-4ccd-9950-67c1da3f787e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-e1db66b4-884d-4ccd-9950-67c1da3f787e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:40.125708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:40.126637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:40.127562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:40.127562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:40.127562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:40.127562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 436/861] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:40.132677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f941350-fdab-474f-bf8c-9552bc73749a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:40.243021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f941350-fdab-474f-bf8c-9552bc73749a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:40.283976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51ccbf-f827-4ee7-b32f-9f2670a61bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:40.283976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51ccbf-f827-4ee7-b32f-9f2670a61bac None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:40.283976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51ccbf-f827-4ee7-b32f-9f2670a61bac None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:40.283976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51ccbf-f827-4ee7-b32f-9f2670a61bac None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:40.284454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51ccbf-f827-4ee7-b32f-9f2670a61bac None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:40.284454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51ccbf-f827-4ee7-b32f-9f2670a61bac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:40.285755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51ccbf-f827-4ee7-b32f-9f2670a61bac None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:40.304236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 426/862] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:40.309731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db66b4-884d-4ccd-9950-67c1da3f787e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:40.337438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:40.419745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db66b4-884d-4ccd-9950-67c1da3f787e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:40.455278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd52970-6ce8-4fc5-a966-9da71838393f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:40.456282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd52970-6ce8-4fc5-a966-9da71838393f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:40.456776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd52970-6ce8-4fc5-a966-9da71838393f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:40.457242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd52970-6ce8-4fc5-a966-9da71838393f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:40.465162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd52970-6ce8-4fc5-a966-9da71838393f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:40.466919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd52970-6ce8-4fc5-a966-9da71838393f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-bbd52970-6ce8-4fc5-a966-9da71838393f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:40.479521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:40.480472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:40.481408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:40.481408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:40.481408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:40.481408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:40.481408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:40.481408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:40.481408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:40.481408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:40.481408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:40.481408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:40.483947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 437/863] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 3 14:39:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:40.489602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51ccbf-f827-4ee7-b32f-9f2670a61bac None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:40.549104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:40.594157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51ccbf-f827-4ee7-b32f-9f2670a61bac None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:40.629351 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca53d85-af6a-4b68-b52f-e39b9c7cdb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:40.630221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca53d85-af6a-4b68-b52f-e39b9c7cdb4d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:40.630712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca53d85-af6a-4b68-b52f-e39b9c7cdb4d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:40.631156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca53d85-af6a-4b68-b52f-e39b9c7cdb4d None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:40.632037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca53d85-af6a-4b68-b52f-e39b9c7cdb4d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:40.632504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca53d85-af6a-4b68-b52f-e39b9c7cdb4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:40.634091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca53d85-af6a-4b68-b52f-e39b9c7cdb4d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:40.653781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 427/864] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 3 14:39:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:40.662807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd52970-6ce8-4fc5-a966-9da71838393f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:40.768281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:40.788962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd52970-6ce8-4fc5-a966-9da71838393f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:40.826489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b42da-9ede-4171-88b1-8ae056116e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:40.827384 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b42da-9ede-4171-88b1-8ae056116e97 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:40.827806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b42da-9ede-4171-88b1-8ae056116e97 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:40.828221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b42da-9ede-4171-88b1-8ae056116e97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:40.857897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a4b6a-9be6-4596-a12c-316726a0f51d None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:40.858692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a4b6a-9be6-4596-a12c-316726a0f51d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:40.859417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a4b6a-9be6-4596-a12c-316726a0f51d None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:40.861564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 428/865] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:40.868023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808265-0b53-4aad-acea-d8ce7007f4ed None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:40.868762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808265-0b53-4aad-acea-d8ce7007f4ed None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:40.869258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808265-0b53-4aad-acea-d8ce7007f4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:40.888429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ea808265-0b53-4aad-acea-d8ce7007f4ed None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:40.889984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ea808265-0b53-4aad-acea-d8ce7007f4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:40.903976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b42da-9ede-4171-88b1-8ae056116e97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:40.905423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b42da-9ede-4171-88b1-8ae056116e97 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:40.965876 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 429/866] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:40.997899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:41.008045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 438/867] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:39:41.017285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea808265-0b53-4aad-acea-d8ce7007f4ed None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:41.067550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:41.119567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea808265-0b53-4aad-acea-d8ce7007f4ed None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:41.155925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e81598-8f08-484c-9775-c6efa740aa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:41.156585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e81598-8f08-484c-9775-c6efa740aa19 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:41.156863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e81598-8f08-484c-9775-c6efa740aa19 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:41.157135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e81598-8f08-484c-9775-c6efa740aa19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:41.173271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e81598-8f08-484c-9775-c6efa740aa19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:41.174767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e81598-8f08-484c-9775-c6efa740aa19 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-a7e81598-8f08-484c-9775-c6efa740aa19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:41.190122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:41.191016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:41.192296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:41.192296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:41.192296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:41.192296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 430/868] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:41.197603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b42da-9ede-4171-88b1-8ae056116e97 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:41.261752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71cbc4-c982-4449-9e1f-9974dfaac948 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:41.262060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71cbc4-c982-4449-9e1f-9974dfaac948 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:41.262297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71cbc4-c982-4449-9e1f-9974dfaac948 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:41.264071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 431/869] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:41.264625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71cbc4-c982-4449-9e1f-9974dfaac948 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:41.307401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b42da-9ede-4171-88b1-8ae056116e97 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:41.341310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9a38-4774-4f13-abe0-9b0c58702ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:41.341955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9a38-4774-4f13-abe0-9b0c58702ec9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:41.342236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9a38-4774-4f13-abe0-9b0c58702ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:41.342507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9a38-4774-4f13-abe0-9b0c58702ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:41.343157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac9a38-4774-4f13-abe0-9b0c58702ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:41.343462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac9a38-4774-4f13-abe0-9b0c58702ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:41.344825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac9a38-4774-4f13-abe0-9b0c58702ec9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:41.357551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 439/870] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:41.357551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade5d68-c3d4-4d12-9f4f-4bda3bba7d3d None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:41.357765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade5d68-c3d4-4d12-9f4f-4bda3bba7d3d None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:41.358410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade5d68-c3d4-4d12-9f4f-4bda3bba7d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:41.369372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:41.374047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71cbc4-c982-4449-9e1f-9974dfaac948 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:41.377132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-4ade5d68-c3d4-4d12-9f4f-4bda3bba7d3d None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:41.378357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4ade5d68-c3d4-4d12-9f4f-4bda3bba7d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:41.409931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbdc58-cfbd-4f6a-8c61-4f30766bfb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:41.410541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbdc58-cfbd-4f6a-8c61-4f30766bfb23 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:41.410817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbdc58-cfbd-4f6a-8c61-4f30766bfb23 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:41.411083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbdc58-cfbd-4f6a-8c61-4f30766bfb23 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:41.411725 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbdc58-cfbd-4f6a-8c61-4f30766bfb23 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:41.413696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbdc58-cfbd-4f6a-8c61-4f30766bfb23 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:41.424739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:41.430807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 432/871] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:39:41 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:39:41.430807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbdc58-cfbd-4f6a-8c61-4f30766bfb23 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:41.465249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 440/872] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:41.560332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbdc58-cfbd-4f6a-8c61-4f30766bfb23 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:41.598266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29ef29-66c9-4449-b006-bde5905e8e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:41.599154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29ef29-66c9-4449-b006-bde5905e8e98 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:41.599788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29ef29-66c9-4449-b006-bde5905e8e98 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:41.600212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29ef29-66c9-4449-b006-bde5905e8e98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:41.607930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29ef29-66c9-4449-b006-bde5905e8e98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:41.609642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29ef29-66c9-4449-b006-bde5905e8e98 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-cb29ef29-66c9-4449-b006-bde5905e8e98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:41.622484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:41.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:41.624730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:41.624730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:41.624730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:41.624730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:41.624730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:41.624730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:41.624730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:41.624730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:41.626747 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 433/873] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:41.632645 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade5d68-c3d4-4d12-9f4f-4bda3bba7d3d None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:41.738077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29ef29-66c9-4449-b006-bde5905e8e98 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:41.752467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade5d68-c3d4-4d12-9f4f-4bda3bba7d3d None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:41.762685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:41.798745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aff8ec-4c14-40af-8ec0-574da412cb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:41.799511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aff8ec-4c14-40af-8ec0-574da412cb05 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:41.799908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aff8ec-4c14-40af-8ec0-574da412cb05 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:41.800187 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aff8ec-4c14-40af-8ec0-574da412cb05 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:41.800885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aff8ec-4c14-40af-8ec0-574da412cb05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:41.801193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aff8ec-4c14-40af-8ec0-574da412cb05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:41.803281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aff8ec-4c14-40af-8ec0-574da412cb05 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:41.818855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 441/874] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:41.834877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aff8ec-4c14-40af-8ec0-574da412cb05 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:41.861776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29ef29-66c9-4449-b006-bde5905e8e98 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:41.878632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:41.899670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71966f27-2413-40c8-9075-cca1b418a0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:41.900365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71966f27-2413-40c8-9075-cca1b418a0cf None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:41.900711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71966f27-2413-40c8-9075-cca1b418a0cf None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:41.901080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71966f27-2413-40c8-9075-cca1b418a0cf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:41.901798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71966f27-2413-40c8-9075-cca1b418a0cf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:41.903420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71966f27-2413-40c8-9075-cca1b418a0cf None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-71966f27-2413-40c8-9075-cca1b418a0cf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:41.912910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:41.914031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:41.914878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 434/875] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:41.922128 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71966f27-2413-40c8-9075-cca1b418a0cf None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:41.974674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aff8ec-4c14-40af-8ec0-574da412cb05 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:42.009906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4601148-153f-4c57-ac70-fbb35aa6dbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:42.010564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4601148-153f-4c57-ac70-fbb35aa6dbc9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:42.011029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4601148-153f-4c57-ac70-fbb35aa6dbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:42.011158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4601148-153f-4c57-ac70-fbb35aa6dbc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:42.011936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4601148-153f-4c57-ac70-fbb35aa6dbc9 None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=fd0db9d2a753490cb8473229a8f6fe15) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:42.012262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4601148-153f-4c57-ac70-fbb35aa6dbc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:42.013764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4601148-153f-4c57-ac70-fbb35aa6dbc9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:42.025669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 442/876] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:41 2022] GET /identity/v3/role_assignments?user.id=fd0db9d2a753490cb8473229a8f6fe15&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:42.038680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:42.043229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27451bd-b162-4489-8d65-cb989583edb3 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:42.043512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27451bd-b162-4489-8d65-cb989583edb3 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:42.043774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27451bd-b162-4489-8d65-cb989583edb3 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:42.049242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 443/877] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:42.050972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71537-8ecc-430c-b6ac-b87910357c5b None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:42.051255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71537-8ecc-430c-b6ac-b87910357c5b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:42.051527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71537-8ecc-430c-b6ac-b87910357c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:42.067495 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71966f27-2413-40c8-9075-cca1b418a0cf None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:42.070450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-edc71537-8ecc-430c-b6ac-b87910357c5b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:42.071650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-edc71537-8ecc-430c-b6ac-b87910357c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:42.111837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ec786-f6d8-4d26-b3ad-a86be04a4dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:42.112490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ec786-f6d8-4d26-b3ad-a86be04a4dc4 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:42.112766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ec786-f6d8-4d26-b3ad-a86be04a4dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:42.113033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ec786-f6d8-4d26-b3ad-a86be04a4dc4 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:42.113983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ec786-f6d8-4d26-b3ad-a86be04a4dc4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:42.114284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ec786-f6d8-4d26-b3ad-a86be04a4dc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:42.116252 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ec786-f6d8-4d26-b3ad-a86be04a4dc4 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:42.128123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 435/878] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 3 14:39:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:42.129416 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ec786-f6d8-4d26-b3ad-a86be04a4dc4 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:42.184956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 444/879] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:42 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:42.248665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ec786-f6d8-4d26-b3ad-a86be04a4dc4 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:42.285056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9584593-d552-4397-a76e-71fff10ff47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:42.286306 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9584593-d552-4397-a76e-71fff10ff47a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:42.286306 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9584593-d552-4397-a76e-71fff10ff47a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:42.286306 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9584593-d552-4397-a76e-71fff10ff47a None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:42.287486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9584593-d552-4397-a76e-71fff10ff47a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:42.287794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9584593-d552-4397-a76e-71fff10ff47a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:42.289160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9584593-d552-4397-a76e-71fff10ff47a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:42.296273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 436/880] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 3 14:39:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:42.463853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc71537-8ecc-430c-b6ac-b87910357c5b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:42.490499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:42.569463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc71537-8ecc-430c-b6ac-b87910357c5b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:42.604485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa415393-d454-4404-a201-c1fdf58d5e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:42.605087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa415393-d454-4404-a201-c1fdf58d5e9d None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:42.605359 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa415393-d454-4404-a201-c1fdf58d5e9d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:42.605654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa415393-d454-4404-a201-c1fdf58d5e9d None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:42.606319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa415393-d454-4404-a201-c1fdf58d5e9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:42.607743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa415393-d454-4404-a201-c1fdf58d5e9d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:42.626046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 445/881] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:39:42 2022] POST /identity/v3/services => generated 269 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:39:43.689349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff2268-f120-4f57-818a-a8d83260bb7e None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:43.689992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff2268-f120-4f57-818a-a8d83260bb7e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:43.689992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff2268-f120-4f57-818a-a8d83260bb7e None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:43.691517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 437/882] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:43.697205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395c840-7f63-4e14-a5ec-e618440e5c82 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:43.698090 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395c840-7f63-4e14-a5ec-e618440e5c82 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:43.698571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395c840-7f63-4e14-a5ec-e618440e5c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:43.717633 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-2395c840-7f63-4e14-a5ec-e618440e5c82 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:43.719130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2395c840-7f63-4e14-a5ec-e618440e5c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:43.793617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 446/883] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:44.068854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff2268-f120-4f57-818a-a8d83260bb7e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:44.085326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:44.174234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff2268-f120-4f57-818a-a8d83260bb7e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:44.211120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce56da0-2873-4846-80f4-30135dd928c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:44.211818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce56da0-2873-4846-80f4-30135dd928c8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.212102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce56da0-2873-4846-80f4-30135dd928c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.212369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce56da0-2873-4846-80f4-30135dd928c8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.212967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce56da0-2873-4846-80f4-30135dd928c8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:44.214413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce56da0-2873-4846-80f4-30135dd928c8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-bce56da0-2873-4846-80f4-30135dd928c8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:44.224521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:44.225681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:44.226568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 438/884] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:39:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:44.234248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395c840-7f63-4e14-a5ec-e618440e5c82 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:44.248168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:44.344486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa7da-bf25-4f72-9e88-c43d0b60c10b None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.345096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa7da-bf25-4f72-9e88-c43d0b60c10b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.345621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa7da-bf25-4f72-9e88-c43d0b60c10b None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.347152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 439/885] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:44.350987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395c840-7f63-4e14-a5ec-e618440e5c82 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:44.352766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e1958-8414-4284-abec-a0cfeeab285f None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.353261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e1958-8414-4284-abec-a0cfeeab285f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.353769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e1958-8414-4284-abec-a0cfeeab285f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.374323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-da9e1958-8414-4284-abec-a0cfeeab285f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:44.375853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-da9e1958-8414-4284-abec-a0cfeeab285f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:44.386005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd18adf-ba88-43d6-a33b-ffecf5ebe51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:44.386839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd18adf-ba88-43d6-a33b-ffecf5ebe51f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.387341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd18adf-ba88-43d6-a33b-ffecf5ebe51f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.387779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd18adf-ba88-43d6-a33b-ffecf5ebe51f None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.388619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd18adf-ba88-43d6-a33b-ffecf5ebe51f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:44.389084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd18adf-ba88-43d6-a33b-ffecf5ebe51f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:44.390894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd18adf-ba88-43d6-a33b-ffecf5ebe51f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:44.396813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 447/886] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:39:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:44.402526 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd18adf-ba88-43d6-a33b-ffecf5ebe51f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:44.453611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 440/887] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:44.483133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:44.511386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd18adf-ba88-43d6-a33b-ffecf5ebe51f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:44.550643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a0ea2-49e4-4dba-971c-17398d400b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:44.551281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a0ea2-49e4-4dba-971c-17398d400b29 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.551562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a0ea2-49e4-4dba-971c-17398d400b29 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.551861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a0ea2-49e4-4dba-971c-17398d400b29 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.552541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a0ea2-49e4-4dba-971c-17398d400b29 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:44.552912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a0ea2-49e4-4dba-971c-17398d400b29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:44.554400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a0ea2-49e4-4dba-971c-17398d400b29 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:44.560409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 448/888] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:39:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:44.566429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e1958-8414-4284-abec-a0cfeeab285f None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:44.575249 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:44.651883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c8a34-b20e-4eff-b07f-9abb7cb3010e None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.652680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c8a34-b20e-4eff-b07f-9abb7cb3010e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.653174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c8a34-b20e-4eff-b07f-9abb7cb3010e None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.655613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 449/889] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:44.660929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e526fc-9fb0-4ee5-8d83-24bebedb4c5b None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.661527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e526fc-9fb0-4ee5-8d83-24bebedb4c5b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.662988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e526fc-9fb0-4ee5-8d83-24bebedb4c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.674834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e1958-8414-4284-abec-a0cfeeab285f None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:44.683078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f8e526fc-9fb0-4ee5-8d83-24bebedb4c5b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:44.684531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f8e526fc-9fb0-4ee5-8d83-24bebedb4c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:44.712945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4807d9-1c55-4806-ae3d-89ad36eaa5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:44.713607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4807d9-1c55-4806-ae3d-89ad36eaa5be None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.714071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4807d9-1c55-4806-ae3d-89ad36eaa5be None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.714482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4807d9-1c55-4806-ae3d-89ad36eaa5be None admin] PATH_INFO: `/v3/endpoints` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.715397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4807d9-1c55-4806-ae3d-89ad36eaa5be None admin] RBAC: Adding query filter params (interface=public, service_id=76fa8a8fd82f4a82ba583c466b142ff4, region_id=RegionOne) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:44.715972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4807d9-1c55-4806-ae3d-89ad36eaa5be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:44.717782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4807d9-1c55-4806-ae3d-89ad36eaa5be None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:44.725346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 441/890] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 3 14:39:44 2022] GET /identity/v3/endpoints?service_id=76fa8a8fd82f4a82ba583c466b142ff4&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:44.726693 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4807d9-1c55-4806-ae3d-89ad36eaa5be None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:44.767264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 450/891] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:44.768778 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e526fc-9fb0-4ee5-8d83-24bebedb4c5b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:44.807983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:44.846670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4807d9-1c55-4806-ae3d-89ad36eaa5be None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:44.853835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:44.883171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40838e1-07fc-4a79-af1e-3a87cac1f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:44.883801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40838e1-07fc-4a79-af1e-3a87cac1f03b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.884087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40838e1-07fc-4a79-af1e-3a87cac1f03b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.884358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40838e1-07fc-4a79-af1e-3a87cac1f03b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.891968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e526fc-9fb0-4ee5-8d83-24bebedb4c5b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:44.895162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40838e1-07fc-4a79-af1e-3a87cac1f03b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-f40838e1-07fc-4a79-af1e-3a87cac1f03b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:44.900474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:44.901383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 442/892] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 3 14:39:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:44.906194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e000d-96f8-4e92-8cab-4285c667ced1 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.908075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e000d-96f8-4e92-8cab-4285c667ced1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.908075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e000d-96f8-4e92-8cab-4285c667ced1 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.908075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 443/893] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:44.908765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e000d-96f8-4e92-8cab-4285c667ced1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:44.931988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0af54-137b-4596-8bd8-7cd93206b1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:44.932602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0af54-137b-4596-8bd8-7cd93206b1e5 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.932884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0af54-137b-4596-8bd8-7cd93206b1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.933159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0af54-137b-4596-8bd8-7cd93206b1e5 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.933809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0af54-137b-4596-8bd8-7cd93206b1e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:44.935189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0af54-137b-4596-8bd8-7cd93206b1e5 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:44.949525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 451/894] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 3 14:39:44 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:39:44.949525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca37d0-3313-4a8d-86a6-9d1d3d78d4f5 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:44.949525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca37d0-3313-4a8d-86a6-9d1d3d78d4f5 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:44.949525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca37d0-3313-4a8d-86a6-9d1d3d78d4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:44.963484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5bca37d0-3313-4a8d-86a6-9d1d3d78d4f5 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:44.964900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5bca37d0-3313-4a8d-86a6-9d1d3d78d4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:45.047352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 452/895] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:45.048565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca37d0-3313-4a8d-86a6-9d1d3d78d4f5 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:45.063328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e000d-96f8-4e92-8cab-4285c667ced1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:45.106858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:45.116135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec030a9b-cf83-48b6-b8e7-98bc2d8b9ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:45.116748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec030a9b-cf83-48b6-b8e7-98bc2d8b9ff1 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:45.117028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec030a9b-cf83-48b6-b8e7-98bc2d8b9ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:45.117299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec030a9b-cf83-48b6-b8e7-98bc2d8b9ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:45.118579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec030a9b-cf83-48b6-b8e7-98bc2d8b9ff1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:45.119084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec030a9b-cf83-48b6-b8e7-98bc2d8b9ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:45.120607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec030a9b-cf83-48b6-b8e7-98bc2d8b9ff1 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:45.131019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 444/896] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 3 14:39:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:45.136919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec030a9b-cf83-48b6-b8e7-98bc2d8b9ff1 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:45.138706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:45.179580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca37d0-3313-4a8d-86a6-9d1d3d78d4f5 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:45.214991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c8b05-7b3d-4391-90f3-6783b90aaf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:45.215867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8b05-7b3d-4391-90f3-6783b90aaf34 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:45.216303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8b05-7b3d-4391-90f3-6783b90aaf34 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:45.216727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8b05-7b3d-4391-90f3-6783b90aaf34 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:45.217554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c8b05-7b3d-4391-90f3-6783b90aaf34 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:45.219076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c8b05-7b3d-4391-90f3-6783b90aaf34 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:45.237612 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 453/897] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:39:45 2022] POST /identity/v3/services => generated 241 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:39:45.277087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec030a9b-cf83-48b6-b8e7-98bc2d8b9ff1 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:45.324000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059e0dd-044a-445f-a6ab-5286567264e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:45.324714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059e0dd-044a-445f-a6ab-5286567264e6 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:45.325000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059e0dd-044a-445f-a6ab-5286567264e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:45.325275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059e0dd-044a-445f-a6ab-5286567264e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:45.326889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c8b05-7b3d-4391-90f3-6783b90aaf34 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:45.339029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059e0dd-044a-445f-a6ab-5286567264e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:45.340703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059e0dd-044a-445f-a6ab-5286567264e6 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-1059e0dd-044a-445f-a6ab-5286567264e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:45.359715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:45.360739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:45.361681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:45.361681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:45.361681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:45.361681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 445/898] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:45.368025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059e0dd-044a-445f-a6ab-5286567264e6 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:45.414517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:45.454183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c8b05-7b3d-4391-90f3-6783b90aaf34 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:45.481081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059e0dd-044a-445f-a6ab-5286567264e6 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:45.484895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:45.506798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593462b9-4956-4ce3-9709-913bb72cc601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:45.507659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593462b9-4956-4ce3-9709-913bb72cc601 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:45.507796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593462b9-4956-4ce3-9709-913bb72cc601 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:45.507975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593462b9-4956-4ce3-9709-913bb72cc601 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:45.508574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593462b9-4956-4ce3-9709-913bb72cc601 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:45.509987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593462b9-4956-4ce3-9709-913bb72cc601 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-593462b9-4956-4ce3-9709-913bb72cc601 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:45.519524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:45.520672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:45.521812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 454/899] 10.222.0.46 () {56 vars in 1132 bytes} [Mon Oct 3 14:39:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:45.526844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593462b9-4956-4ce3-9709-913bb72cc601 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:45.533819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b9dd8-ca15-4cf9-a0db-618fc2d6f76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:45.535872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b9dd8-ca15-4cf9-a0db-618fc2d6f76d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:45.535872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b9dd8-ca15-4cf9-a0db-618fc2d6f76d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:45.535872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b9dd8-ca15-4cf9-a0db-618fc2d6f76d None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:45.541458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b9dd8-ca15-4cf9-a0db-618fc2d6f76d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:45.541625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b9dd8-ca15-4cf9-a0db-618fc2d6f76d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:45.543020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b9dd8-ca15-4cf9-a0db-618fc2d6f76d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:45.557655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 446/900] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:45.563170 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b9dd8-ca15-4cf9-a0db-618fc2d6f76d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:45.669764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:45.673926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b9dd8-ca15-4cf9-a0db-618fc2d6f76d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:45.675600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593462b9-4956-4ce3-9709-913bb72cc601 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:45.711963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4645f9-0300-43fc-8d7d-55625100b9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:45.712692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4645f9-0300-43fc-8d7d-55625100b9e7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:45.713109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4645f9-0300-43fc-8d7d-55625100b9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:45.713712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4645f9-0300-43fc-8d7d-55625100b9e7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:45.721146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4645f9-0300-43fc-8d7d-55625100b9e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:45.722691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4645f9-0300-43fc-8d7d-55625100b9e7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:45.727781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb49998-94fe-463c-bb82-72cb6a367f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:45.728639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb49998-94fe-463c-bb82-72cb6a367f51 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:45.729082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb49998-94fe-463c-bb82-72cb6a367f51 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:45.729528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb49998-94fe-463c-bb82-72cb6a367f51 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:45.730350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb49998-94fe-463c-bb82-72cb6a367f51 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:45.731108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb49998-94fe-463c-bb82-72cb6a367f51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:45.732624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb49998-94fe-463c-bb82-72cb6a367f51 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-ea4645f9-0300-43fc-8d7d-55625100b9e7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:45.736379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:45.737515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:45.738831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:45.738831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:45.738831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:45.738831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:45.738831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:45.738831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:45.738831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:45.738831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:45.738831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:45.738831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:45.745673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 447/902] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 3 14:39:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:45.745822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 455/902] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:45.746807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4645f9-0300-43fc-8d7d-55625100b9e7 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:45.747371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb49998-94fe-463c-bb82-72cb6a367f51 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:45.813466 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:45.869621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb49998-94fe-463c-bb82-72cb6a367f51 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:45.878104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4645f9-0300-43fc-8d7d-55625100b9e7 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:45.905675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b7498-4680-4f4e-9f61-8023dafc2d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:45.906377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b7498-4680-4f4e-9f61-8023dafc2d44 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:45.906611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b7498-4680-4f4e-9f61-8023dafc2d44 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:45.906890 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b7498-4680-4f4e-9f61-8023dafc2d44 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:45.907521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b7498-4680-4f4e-9f61-8023dafc2d44 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:45.907878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b7498-4680-4f4e-9f61-8023dafc2d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:45.909304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b7498-4680-4f4e-9f61-8023dafc2d44 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:45.918977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 456/903] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:45.920022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b7498-4680-4f4e-9f61-8023dafc2d44 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:45.929980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b885739-b807-43d8-b55d-8783b9cea522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:45.930878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b885739-b807-43d8-b55d-8783b9cea522 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:45.931809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b885739-b807-43d8-b55d-8783b9cea522 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:45.932308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b885739-b807-43d8-b55d-8783b9cea522 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:45.933688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b885739-b807-43d8-b55d-8783b9cea522 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:45.934467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b885739-b807-43d8-b55d-8783b9cea522 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:45.936367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b885739-b807-43d8-b55d-8783b9cea522 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:45.960767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 448/904] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 3 14:39:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:45.966574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b885739-b807-43d8-b55d-8783b9cea522 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:45.992997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:46.031401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b7498-4680-4f4e-9f61-8023dafc2d44 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:46.046989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:46.071061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754819-eba9-49aa-bde6-197eddd4dc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:46.071826 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754819-eba9-49aa-bde6-197eddd4dc5e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:46.072195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754819-eba9-49aa-bde6-197eddd4dc5e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:46.072542 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754819-eba9-49aa-bde6-197eddd4dc5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:46.073411 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21754819-eba9-49aa-bde6-197eddd4dc5e None admin] RBAC: Adding query filter params (interface=public, service_id=800a3475e3a7432fabc27807f3e7b2c8, region_id=RegionOne) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:46.073895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21754819-eba9-49aa-bde6-197eddd4dc5e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:46.075437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21754819-eba9-49aa-bde6-197eddd4dc5e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:46.086114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 457/905] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 3 14:39:45 2022] GET /identity/v3/endpoints?service_id=800a3475e3a7432fabc27807f3e7b2c8&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:46.087207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754819-eba9-49aa-bde6-197eddd4dc5e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:46.106900 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b885739-b807-43d8-b55d-8783b9cea522 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:46.150111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645b5d6-fa46-4a3d-9955-868bad2af18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:46.150889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645b5d6-fa46-4a3d-9955-868bad2af18f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:46.151260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645b5d6-fa46-4a3d-9955-868bad2af18f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:46.151635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645b5d6-fa46-4a3d-9955-868bad2af18f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:46.211825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754819-eba9-49aa-bde6-197eddd4dc5e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:46.228212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645b5d6-fa46-4a3d-9955-868bad2af18f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:46.229851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645b5d6-fa46-4a3d-9955-868bad2af18f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:46.249421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:46.250424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765fc5f-4486-49ee-8b09-4162ce0e291d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:46.251249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765fc5f-4486-49ee-8b09-4162ce0e291d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:46.251726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765fc5f-4486-49ee-8b09-4162ce0e291d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:46.252214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765fc5f-4486-49ee-8b09-4162ce0e291d None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:46.253009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765fc5f-4486-49ee-8b09-4162ce0e291d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:46.254583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765fc5f-4486-49ee-8b09-4162ce0e291d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:46.265890 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 458/906] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 3 14:39:46 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:39:46.355764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 449/907] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:39:46.361787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765fc5f-4486-49ee-8b09-4162ce0e291d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:46.416746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:46.480200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765fc5f-4486-49ee-8b09-4162ce0e291d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:46.515648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e651a-0e79-4454-a90d-26d91a97b6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:46.516319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e651a-0e79-4454-a90d-26d91a97b6cf None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:46.516602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e651a-0e79-4454-a90d-26d91a97b6cf None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:46.516876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e651a-0e79-4454-a90d-26d91a97b6cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:46.528277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e651a-0e79-4454-a90d-26d91a97b6cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:46.529764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e651a-0e79-4454-a90d-26d91a97b6cf None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-670e651a-0e79-4454-a90d-26d91a97b6cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:46.545208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:46.546238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:46.547114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:46.547114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:46.547114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:46.547114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 459/908] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:46.553387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645b5d6-fa46-4a3d-9955-868bad2af18f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:46.661761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645b5d6-fa46-4a3d-9955-868bad2af18f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:46.680609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:46.706827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e24de7-c575-4e2b-9a24-f42b1f570f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:46.707518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e24de7-c575-4e2b-9a24-f42b1f570f16 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:46.707805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e24de7-c575-4e2b-9a24-f42b1f570f16 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:46.708083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e24de7-c575-4e2b-9a24-f42b1f570f16 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:46.708823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e24de7-c575-4e2b-9a24-f42b1f570f16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:46.709179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e24de7-c575-4e2b-9a24-f42b1f570f16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:46.710595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e24de7-c575-4e2b-9a24-f42b1f570f16 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:46.722885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 450/909] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:46.728480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e651a-0e79-4454-a90d-26d91a97b6cf None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:46.837953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:46.840827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e651a-0e79-4454-a90d-26d91a97b6cf None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:46.877166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bed20-e9a7-495f-a959-9dc0d3e7137a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:46.877832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bed20-e9a7-495f-a959-9dc0d3e7137a None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:46.878115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bed20-e9a7-495f-a959-9dc0d3e7137a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:46.878450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bed20-e9a7-495f-a959-9dc0d3e7137a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:46.885686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bed20-e9a7-495f-a959-9dc0d3e7137a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:46.887138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bed20-e9a7-495f-a959-9dc0d3e7137a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-086bed20-e9a7-495f-a959-9dc0d3e7137a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:46.899554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:46.900427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:46.901395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:46.901395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:46.901395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:46.901395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:46.901395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:46.901395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:46.901395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:46.901395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:46.901395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 460/910] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:46.907003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e24de7-c575-4e2b-9a24-f42b1f570f16 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:47.018929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e24de7-c575-4e2b-9a24-f42b1f570f16 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:47.063392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de094e2-c127-457b-80c6-371c43ff8a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:47.064014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de094e2-c127-457b-80c6-371c43ff8a4b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:47.064297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de094e2-c127-457b-80c6-371c43ff8a4b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:47.064571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de094e2-c127-457b-80c6-371c43ff8a4b None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:47.065359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de094e2-c127-457b-80c6-371c43ff8a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:47.065700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de094e2-c127-457b-80c6-371c43ff8a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:47.067141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de094e2-c127-457b-80c6-371c43ff8a4b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:47.080924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 451/911] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:47.087689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bed20-e9a7-495f-a959-9dc0d3e7137a None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:47.150792 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793dbe0-adf6-4727-bcc3-453ff064c479 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:47.151077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793dbe0-adf6-4727-bcc3-453ff064c479 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:47.151271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793dbe0-adf6-4727-bcc3-453ff064c479 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:47.152841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 452/912] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:47.157896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437ec47-8e45-4975-8753-3ffd21bd6c6a None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:47.158177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437ec47-8e45-4975-8753-3ffd21bd6c6a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:47.158453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437ec47-8e45-4975-8753-3ffd21bd6c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:47.178441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-4437ec47-8e45-4975-8753-3ffd21bd6c6a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:47.179745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4437ec47-8e45-4975-8753-3ffd21bd6c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:47.201352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bed20-e9a7-495f-a959-9dc0d3e7137a None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:47.216617 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:47.248577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32635665-9ccf-41c8-9a3e-f8c5f7b99e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:47.249639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32635665-9ccf-41c8-9a3e-f8c5f7b99e6e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:47.250358 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32635665-9ccf-41c8-9a3e-f8c5f7b99e6e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:47.251055 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32635665-9ccf-41c8-9a3e-f8c5f7b99e6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:47.251950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32635665-9ccf-41c8-9a3e-f8c5f7b99e6e None admin] RBAC: Adding query filter params (role.id=662ff20775c14ad69b5a84324701fcf1, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=fd0db9d2a753490cb8473229a8f6fe15) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:47.252529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32635665-9ccf-41c8-9a3e-f8c5f7b99e6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:47.254816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32635665-9ccf-41c8-9a3e-f8c5f7b99e6e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:47.261717 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 461/914] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:47 2022] GET /identity/v3/role_assignments?user.id=fd0db9d2a753490cb8473229a8f6fe15&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=662ff20775c14ad69b5a84324701fcf1 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:47.262514 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 453/914] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:47.519244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a221cd3-1eab-44a0-825f-06066ce27318 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:47.519616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a221cd3-1eab-44a0-825f-06066ce27318 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:47.519876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a221cd3-1eab-44a0-825f-06066ce27318 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:47.522031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 462/915] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:47.527382 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab06e8-1758-44c0-bf61-f5b8997340f0 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:47.527918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab06e8-1758-44c0-bf61-f5b8997340f0 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:47.528413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab06e8-1758-44c0-bf61-f5b8997340f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:47.537888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a221cd3-1eab-44a0-825f-06066ce27318 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:47.548158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-00ab06e8-1758-44c0-bf61-f5b8997340f0 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:47.549659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-00ab06e8-1758-44c0-bf61-f5b8997340f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:47.561622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:47.627000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 454/916] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:47.659855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a221cd3-1eab-44a0-825f-06066ce27318 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:47.695100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ee0b8-86b8-4e7a-99ab-c8ea1154a367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:47.695735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ee0b8-86b8-4e7a-99ab-c8ea1154a367 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:47.696016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ee0b8-86b8-4e7a-99ab-c8ea1154a367 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:47.696287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ee0b8-86b8-4e7a-99ab-c8ea1154a367 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:47.696871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ee0b8-86b8-4e7a-99ab-c8ea1154a367 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:47.698267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ee0b8-86b8-4e7a-99ab-c8ea1154a367 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-438ee0b8-86b8-4e7a-99ab-c8ea1154a367 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:47.707194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:47.708188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:47.709135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 463/917] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:39:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:47.714397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab06e8-1758-44c0-bf61-f5b8997340f0 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:47.810996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:47.814777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab06e8-1758-44c0-bf61-f5b8997340f0 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:47.854828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb2633-5702-4e8c-9acb-24d3670bba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:47.855448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb2633-5702-4e8c-9acb-24d3670bba94 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:47.855731 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb2633-5702-4e8c-9acb-24d3670bba94 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:47.856005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb2633-5702-4e8c-9acb-24d3670bba94 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:47.856643 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb2633-5702-4e8c-9acb-24d3670bba94 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:47.856950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb2633-5702-4e8c-9acb-24d3670bba94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:47.858345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb2633-5702-4e8c-9acb-24d3670bba94 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:47.863676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 455/918] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:39:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:47.869127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ee0b8-86b8-4e7a-99ab-c8ea1154a367 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:47.921177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb2633-5702-4e8c-9acb-24d3670bba94 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:47.963855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:47.990154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ee0b8-86b8-4e7a-99ab-c8ea1154a367 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:48.028242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0ee48-98f2-45a8-ab72-6b9589b5a6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:48.028862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0ee48-98f2-45a8-ab72-6b9589b5a6fd None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:48.029159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0ee48-98f2-45a8-ab72-6b9589b5a6fd None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:48.029408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0ee48-98f2-45a8-ab72-6b9589b5a6fd None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:48.030071 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0ee48-98f2-45a8-ab72-6b9589b5a6fd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:48.030375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0ee48-98f2-45a8-ab72-6b9589b5a6fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:48.031750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0ee48-98f2-45a8-ab72-6b9589b5a6fd None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:48.037705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 464/919] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 3 14:39:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:48.042951 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0ee48-98f2-45a8-ab72-6b9589b5a6fd None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:48.053781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb2633-5702-4e8c-9acb-24d3670bba94 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:48.102617 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d1bcb-8949-4b33-b194-04eb5a43f006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:48.103860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d1bcb-8949-4b33-b194-04eb5a43f006 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:48.104618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d1bcb-8949-4b33-b194-04eb5a43f006 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:48.105519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d1bcb-8949-4b33-b194-04eb5a43f006 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:48.106618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d1bcb-8949-4b33-b194-04eb5a43f006 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:48.108894 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d1bcb-8949-4b33-b194-04eb5a43f006 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-1d7d1bcb-8949-4b33-b194-04eb5a43f006 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:48.121177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:48.122231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:48.123078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 456/920] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:48.129832 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d1bcb-8949-4b33-b194-04eb5a43f006 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:48.158893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0ee48-98f2-45a8-ab72-6b9589b5a6fd None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:48.170882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:48.204994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b54d2-34da-450d-ab53-810457d8e774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:48.205697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b54d2-34da-450d-ab53-810457d8e774 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:48.206025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b54d2-34da-450d-ab53-810457d8e774 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:48.206296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b54d2-34da-450d-ab53-810457d8e774 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:48.207361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b54d2-34da-450d-ab53-810457d8e774 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:48.209255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b54d2-34da-450d-ab53-810457d8e774 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:48.237584 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 465/921] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:39:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:39:48.271868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d1bcb-8949-4b33-b194-04eb5a43f006 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:48.330483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6844600-65c8-4aef-9b51-4c43a44db501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:48.331552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6844600-65c8-4aef-9b51-4c43a44db501 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:48.332108 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6844600-65c8-4aef-9b51-4c43a44db501 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:48.332612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6844600-65c8-4aef-9b51-4c43a44db501 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:48.333571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6844600-65c8-4aef-9b51-4c43a44db501 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:48.334097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6844600-65c8-4aef-9b51-4c43a44db501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:48.335756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6844600-65c8-4aef-9b51-4c43a44db501 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:48.341925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 457/922] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 3 14:39:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:48.348088 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b54d2-34da-450d-ab53-810457d8e774 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:48.351555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:48.468198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b54d2-34da-450d-ab53-810457d8e774 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:48.503708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849b7cb-06c6-473a-9565-ea0b09276850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:48.504320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849b7cb-06c6-473a-9565-ea0b09276850 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:48.504605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849b7cb-06c6-473a-9565-ea0b09276850 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:48.504874 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849b7cb-06c6-473a-9565-ea0b09276850 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:48.505573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849b7cb-06c6-473a-9565-ea0b09276850 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:48.505888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849b7cb-06c6-473a-9565-ea0b09276850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:48.507385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849b7cb-06c6-473a-9565-ea0b09276850 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:48.512828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 466/923] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 3 14:39:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:48.519021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6844600-65c8-4aef-9b51-4c43a44db501 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:48.549362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea040cf-b17d-4a9b-8a2c-86451dc1c01b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:48.549687 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea040cf-b17d-4a9b-8a2c-86451dc1c01b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:48.549989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea040cf-b17d-4a9b-8a2c-86451dc1c01b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:48.551532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 467/924] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:48.557128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87654cc-28b3-42d2-aee7-d2b1a6d40cd6 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:48.557421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87654cc-28b3-42d2-aee7-d2b1a6d40cd6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:48.557727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87654cc-28b3-42d2-aee7-d2b1a6d40cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:48.575851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f87654cc-28b3-42d2-aee7-d2b1a6d40cd6 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:48.577036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f87654cc-28b3-42d2-aee7-d2b1a6d40cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:48.634304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:48.638877 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6844600-65c8-4aef-9b51-4c43a44db501 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:48.644660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:48.653327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 468/925] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:48.682640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62af90-efb7-4734-844a-07808ccbdd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:48.683642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62af90-efb7-4734-844a-07808ccbdd2c None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:48.684135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62af90-efb7-4734-844a-07808ccbdd2c None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:48.684620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62af90-efb7-4734-844a-07808ccbdd2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:48.685817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62af90-efb7-4734-844a-07808ccbdd2c None admin] RBAC: Adding query filter params (interface=public, service_id=607f0da4bf9b46fa8e9b18c8a0532fff, region_id=RegionOne) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:48.686333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62af90-efb7-4734-844a-07808ccbdd2c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:48.687901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62af90-efb7-4734-844a-07808ccbdd2c None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:48.698748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 458/926] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 3 14:39:48 2022] GET /identity/v3/endpoints?service_id=607f0da4bf9b46fa8e9b18c8a0532fff&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:48.702574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87654cc-28b3-42d2-aee7-d2b1a6d40cd6 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:48.805801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87654cc-28b3-42d2-aee7-d2b1a6d40cd6 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:48.842056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1a6be-8333-42d7-ba68-6f1c45c9982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:48.843059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1a6be-8333-42d7-ba68-6f1c45c9982c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:48.843535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1a6be-8333-42d7-ba68-6f1c45c9982c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:48.844007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1a6be-8333-42d7-ba68-6f1c45c9982c None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:48.844801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1a6be-8333-42d7-ba68-6f1c45c9982c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:48.846482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1a6be-8333-42d7-ba68-6f1c45c9982c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:48.857642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 469/927] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 3 14:39:48 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:39:48.930060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62af90-efb7-4734-844a-07808ccbdd2c None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:49.047254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62af90-efb7-4734-844a-07808ccbdd2c None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:49.082958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e456ec9-2a0e-44db-9ee8-cf9da803e10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:49.084492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e456ec9-2a0e-44db-9ee8-cf9da803e10d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:49.084968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e456ec9-2a0e-44db-9ee8-cf9da803e10d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:49.085440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e456ec9-2a0e-44db-9ee8-cf9da803e10d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:49.086312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e456ec9-2a0e-44db-9ee8-cf9da803e10d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:49.087860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e456ec9-2a0e-44db-9ee8-cf9da803e10d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-9e456ec9-2a0e-44db-9ee8-cf9da803e10d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:49.097338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:49.098290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 14:39:49.099236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:49.102007 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 459/928] 10.222.0.46 () {56 vars in 1132 bytes} [Mon Oct 3 14:39:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:49.109609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1a6be-8333-42d7-ba68-6f1c45c9982c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:49.172551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:49.214950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1a6be-8333-42d7-ba68-6f1c45c9982c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:49.250736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e92183-f3ec-4ef0-b948-97f6dd5c0912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:49.251361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e92183-f3ec-4ef0-b948-97f6dd5c0912 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:49.251639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e92183-f3ec-4ef0-b948-97f6dd5c0912 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:49.251908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e92183-f3ec-4ef0-b948-97f6dd5c0912 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:49.252544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e92183-f3ec-4ef0-b948-97f6dd5c0912 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:49.252846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e92183-f3ec-4ef0-b948-97f6dd5c0912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:49.254273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e92183-f3ec-4ef0-b948-97f6dd5c0912 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:49.260428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 470/929] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:49.266316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e456ec9-2a0e-44db-9ee8-cf9da803e10d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:49.284388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:49.387299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e456ec9-2a0e-44db-9ee8-cf9da803e10d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:49.423025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924683e7-e286-459b-9f6c-ee17bfa90ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:49.423630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924683e7-e286-459b-9f6c-ee17bfa90ad8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:49.423911 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924683e7-e286-459b-9f6c-ee17bfa90ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:49.424184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924683e7-e286-459b-9f6c-ee17bfa90ad8 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:49.424831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924683e7-e286-459b-9f6c-ee17bfa90ad8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:49.425133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924683e7-e286-459b-9f6c-ee17bfa90ad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:49.426530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924683e7-e286-459b-9f6c-ee17bfa90ad8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:49.432009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 460/930] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:49.437691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e92183-f3ec-4ef0-b948-97f6dd5c0912 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:49.458469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deec4b2-ad6d-4bfd-a2b1-5bdd96a796cb None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:49.458776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deec4b2-ad6d-4bfd-a2b1-5bdd96a796cb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:49.459023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deec4b2-ad6d-4bfd-a2b1-5bdd96a796cb None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:49.460600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 461/931] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:49.465447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c9385-7a42-4af9-8d3d-a07d7372c1ce None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:49.465761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c9385-7a42-4af9-8d3d-a07d7372c1ce None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:49.466038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c9385-7a42-4af9-8d3d-a07d7372c1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:49.484450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-744c9385-7a42-4af9-8d3d-a07d7372c1ce None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:49.485681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-744c9385-7a42-4af9-8d3d-a07d7372c1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:49.545228 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e92183-f3ec-4ef0-b948-97f6dd5c0912 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:49.566300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 462/932] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:49.573961 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:49.583695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2bb18-73ab-4f19-b60c-c81da73b595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:49.584300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2bb18-73ab-4f19-b60c-c81da73b595c None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:49.584574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2bb18-73ab-4f19-b60c-c81da73b595c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:49.584843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2bb18-73ab-4f19-b60c-c81da73b595c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:49.585518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2bb18-73ab-4f19-b60c-c81da73b595c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:49.587179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2bb18-73ab-4f19-b60c-c81da73b595c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:49.611435 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 471/933] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:39:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:39:49.836213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c9385-7a42-4af9-8d3d-a07d7372c1ce None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:49.846335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:49.938939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c9385-7a42-4af9-8d3d-a07d7372c1ce None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:49.980009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189b0fc-2087-443b-910e-07d1e34d4357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:49.980656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b0fc-2087-443b-910e-07d1e34d4357 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:49.980936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b0fc-2087-443b-910e-07d1e34d4357 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:49.981190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b0fc-2087-443b-910e-07d1e34d4357 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:49.992817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189b0fc-2087-443b-910e-07d1e34d4357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-8189b0fc-2087-443b-910e-07d1e34d4357 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:49.997626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:49.998697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:39:49.999709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:50.002455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 463/934] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 3 14:39:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:50.008527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2bb18-73ab-4f19-b60c-c81da73b595c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:50.127201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2bb18-73ab-4f19-b60c-c81da73b595c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:50.162142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3735a-e9a3-476c-b8d8-bd1e38bb55e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:50.162754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3735a-e9a3-476c-b8d8-bd1e38bb55e8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:50.163028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3735a-e9a3-476c-b8d8-bd1e38bb55e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:50.163353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3735a-e9a3-476c-b8d8-bd1e38bb55e8 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:50.164021 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3735a-e9a3-476c-b8d8-bd1e38bb55e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:50.164318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3735a-e9a3-476c-b8d8-bd1e38bb55e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:50.165700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3735a-e9a3-476c-b8d8-bd1e38bb55e8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:50.176078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 472/935] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 3 14:39:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:50.181898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189b0fc-2087-443b-910e-07d1e34d4357 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:50.245673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:50.295953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189b0fc-2087-443b-910e-07d1e34d4357 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:50.330760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8919b9-1515-4464-865a-6b6f57f316fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:50.331396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8919b9-1515-4464-865a-6b6f57f316fc None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:50.331675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8919b9-1515-4464-865a-6b6f57f316fc None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:50.331945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8919b9-1515-4464-865a-6b6f57f316fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:50.343212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8919b9-1515-4464-865a-6b6f57f316fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:50.344692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8919b9-1515-4464-865a-6b6f57f316fc None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-4a8919b9-1515-4464-865a-6b6f57f316fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:50.361860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:50.362838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:50.364493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:50.364493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:50.364493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:50.364493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 464/936] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:50.370567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3735a-e9a3-476c-b8d8-bd1e38bb55e8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:50.415749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:50.474182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3735a-e9a3-476c-b8d8-bd1e38bb55e8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:50.509933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816a965-51d7-4460-8a4e-84ec1e5dac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:50.510919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816a965-51d7-4460-8a4e-84ec1e5dac62 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:50.511414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816a965-51d7-4460-8a4e-84ec1e5dac62 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:50.511879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816a965-51d7-4460-8a4e-84ec1e5dac62 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:50.512758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816a965-51d7-4460-8a4e-84ec1e5dac62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:50.513301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816a965-51d7-4460-8a4e-84ec1e5dac62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:50.514969 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816a965-51d7-4460-8a4e-84ec1e5dac62 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:50.533621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 473/937] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:50.536430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8919b9-1515-4464-865a-6b6f57f316fc None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:50.644813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8919b9-1515-4464-865a-6b6f57f316fc None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:50.661765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:50.668471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4e61f-56d3-443f-a388-070f70c6f339 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:50.669015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4e61f-56d3-443f-a388-070f70c6f339 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:50.669485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4e61f-56d3-443f-a388-070f70c6f339 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:50.672745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 474/938] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:50.676878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e562efe-666e-4847-a8bf-68e3667f3a71 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:50.677403 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e562efe-666e-4847-a8bf-68e3667f3a71 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:50.678045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e562efe-666e-4847-a8bf-68e3667f3a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:50.681216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452de07-bcc0-4d8b-858a-94194b38be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:50.681879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452de07-bcc0-4d8b-858a-94194b38be09 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:50.682165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452de07-bcc0-4d8b-858a-94194b38be09 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:50.682427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452de07-bcc0-4d8b-858a-94194b38be09 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:50.691014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452de07-bcc0-4d8b-858a-94194b38be09 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:50.692971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452de07-bcc0-4d8b-858a-94194b38be09 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:50.700935 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-2e562efe-666e-4847-a8bf-68e3667f3a71 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:50.702484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2e562efe-666e-4847-a8bf-68e3667f3a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-5452de07-bcc0-4d8b-858a-94194b38be09 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:50.704680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:50.709680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:50.710737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 465/939] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 3 14:39:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:50.712292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452de07-bcc0-4d8b-858a-94194b38be09 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:50.794668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 475/940] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:50.831542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452de07-bcc0-4d8b-858a-94194b38be09 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:50.866783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f1135-2063-4b47-b181-26c7ca9bbac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:50.867637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1135-2063-4b47-b181-26c7ca9bbac8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:50.868069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1135-2063-4b47-b181-26c7ca9bbac8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:50.868491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1135-2063-4b47-b181-26c7ca9bbac8 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:50.869392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f1135-2063-4b47-b181-26c7ca9bbac8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:50.869872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f1135-2063-4b47-b181-26c7ca9bbac8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:50.871551 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f1135-2063-4b47-b181-26c7ca9bbac8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:50.890211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 466/941] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 3 14:39:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:50.898613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e562efe-666e-4847-a8bf-68e3667f3a71 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:50.930234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:51.002672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e562efe-666e-4847-a8bf-68e3667f3a71 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:51.038481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d16885-bfda-43ea-bd87-1a2a8a1a964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:51.039103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d16885-bfda-43ea-bd87-1a2a8a1a964c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.039376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d16885-bfda-43ea-bd87-1a2a8a1a964c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.039641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d16885-bfda-43ea-bd87-1a2a8a1a964c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.049624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d16885-bfda-43ea-bd87-1a2a8a1a964c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:51.051070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d16885-bfda-43ea-bd87-1a2a8a1a964c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-21d16885-bfda-43ea-bd87-1a2a8a1a964c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:51.068193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:51.069245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:51.070181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:51.070181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:51.070181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.073860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 476/942] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:51.075711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f1135-2063-4b47-b181-26c7ca9bbac8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:51.094710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d16885-bfda-43ea-bd87-1a2a8a1a964c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:51.117341 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:51.185858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:51.191438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f1135-2063-4b47-b181-26c7ca9bbac8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:51.203178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d16885-bfda-43ea-bd87-1a2a8a1a964c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:51.230053 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731ee97-8959-4270-b92d-948baf0e86fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:51.230955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731ee97-8959-4270-b92d-948baf0e86fe None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.231427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731ee97-8959-4270-b92d-948baf0e86fe None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.231700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731ee97-8959-4270-b92d-948baf0e86fe None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.232346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731ee97-8959-4270-b92d-948baf0e86fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:51.232649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731ee97-8959-4270-b92d-948baf0e86fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:51.234034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731ee97-8959-4270-b92d-948baf0e86fe None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:51.239898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9996169-2812-42f1-8195-3735109cf876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:51.240936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9996169-2812-42f1-8195-3735109cf876 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.241224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9996169-2812-42f1-8195-3735109cf876 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.241505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9996169-2812-42f1-8195-3735109cf876 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.242151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9996169-2812-42f1-8195-3735109cf876 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:51.244010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9996169-2812-42f1-8195-3735109cf876 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:51.250064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 467/943] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:51.250199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WARNING keystone.server.flask.common [None req-b9996169-2812-42f1-8195-3735109cf876 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 03 14:39:51.250282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2dc8b-5015-48c5-8e2f-bf9aa5d43e1b None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.250282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2dc8b-5015-48c5-8e2f-bf9aa5d43e1b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.250282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2dc8b-5015-48c5-8e2f-bf9aa5d43e1b None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.250282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 468/944] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:51.250691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2dc8b-5015-48c5-8e2f-bf9aa5d43e1b None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:51.271723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.sql.core [None req-b9996169-2812-42f1-8195-3735109cf876 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 14:39:51.271723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 03 14:39:51.271723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [parameters: {'id': 'b4c1d243233943439a9e35dd6d608f90', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 03 14:39:51.271723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87831) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-b9996169-2812-42f1-8195-3735109cf876 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b4c1d243233943439a9e35dd6d608f90.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b4c1d243233943439a9e35dd6d608f90. Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application conn.query(q) Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 14:39:51.287968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application result.read() Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 03 14:39:51.288846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application next(self.gen) Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application yield resource Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application next(self.gen) Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise self.value Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application session.commit() Oct 03 14:39:51.289931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self.session.flush() Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application compat.raise_( Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 14:39:51.290832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 14:39:51.292868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application conn.query(q) Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application result.read() Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 14:39:51.295120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [parameters: {'id': 'b4c1d243233943439a9e35dd6d608f90', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:51.298581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:51.299782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b4c1d243233943439a9e35dd6d608f90. Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 477/945] 10.222.0.46 () {60 vars in 1140 bytes} [Mon Oct 3 14:39:51 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748626-bfec-4b21-b235-654d21a30a09 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.301519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748626-bfec-4b21-b235-654d21a30a09 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.307947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748626-bfec-4b21-b235-654d21a30a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.314127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-96748626-bfec-4b21-b235-654d21a30a09 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:51.315817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-96748626-bfec-4b21-b235-654d21a30a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:51.363075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:51.372227 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2dc8b-5015-48c5-8e2f-bf9aa5d43e1b None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:51.388632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 478/946] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:51.398487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96748626-bfec-4b21-b235-654d21a30a09 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:51.412691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9c182-55d3-4b40-b2f3-308ff817565d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:51.413518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9c182-55d3-4b40-b2f3-308ff817565d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.413978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9c182-55d3-4b40-b2f3-308ff817565d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.414325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9c182-55d3-4b40-b2f3-308ff817565d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.421888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9c182-55d3-4b40-b2f3-308ff817565d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:51.423279 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9c182-55d3-4b40-b2f3-308ff817565d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-92a9c182-55d3-4b40-b2f3-308ff817565d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:51.435193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:51.436148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:51.437056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:51.437056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:51.437056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:51.437056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:51.437056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:51.437056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:51.437056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:51.437056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:51.441119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 469/947] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:51.445775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9c182-55d3-4b40-b2f3-308ff817565d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:51.491357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:51.534168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96748626-bfec-4b21-b235-654d21a30a09 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:51.562919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9c182-55d3-4b40-b2f3-308ff817565d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:51.579163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed00c2d-d4d3-4c8e-8570-480a207a4930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:51.579995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed00c2d-d4d3-4c8e-8570-480a207a4930 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.580644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed00c2d-d4d3-4c8e-8570-480a207a4930 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.581442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed00c2d-d4d3-4c8e-8570-480a207a4930 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.590031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed00c2d-d4d3-4c8e-8570-480a207a4930 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:51.592782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed00c2d-d4d3-4c8e-8570-480a207a4930 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:51.600364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d940156-d6ca-4ba1-bd27-3c4fd95ea24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:51.601395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d940156-d6ca-4ba1-bd27-3c4fd95ea24d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.601879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d940156-d6ca-4ba1-bd27-3c4fd95ea24d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.602261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d940156-d6ca-4ba1-bd27-3c4fd95ea24d None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.602968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d940156-d6ca-4ba1-bd27-3c4fd95ea24d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:51.603275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d940156-d6ca-4ba1-bd27-3c4fd95ea24d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:51.604707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d940156-d6ca-4ba1-bd27-3c4fd95ea24d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-5ed00c2d-d4d3-4c8e-8570-480a207a4930 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:51.608019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:51.609028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:51.610203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:51.610203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:51.610203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:51.610203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:51.610203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:51.610203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:51.610203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:51.610203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.612672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 479/948] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:51.618117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed00c2d-d4d3-4c8e-8570-480a207a4930 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:51.618812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 470/949] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:51.627262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d940156-d6ca-4ba1-bd27-3c4fd95ea24d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:51.647413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:51.733803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed00c2d-d4d3-4c8e-8570-480a207a4930 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:51.753762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d940156-d6ca-4ba1-bd27-3c4fd95ea24d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:51.763664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:51.771113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e413e2-3e33-4409-945c-b5958ff38207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:51.771719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e413e2-3e33-4409-945c-b5958ff38207 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.772022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e413e2-3e33-4409-945c-b5958ff38207 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.772250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e413e2-3e33-4409-945c-b5958ff38207 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.772940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e413e2-3e33-4409-945c-b5958ff38207 None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:51.773197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e413e2-3e33-4409-945c-b5958ff38207 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:51.774639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e413e2-3e33-4409-945c-b5958ff38207 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:51.791468 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99b76f-0b9f-403a-b94d-8648966f6c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:51.792263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b76f-0b9f-403a-b94d-8648966f6c39 None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.792543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b76f-0b9f-403a-b94d-8648966f6c39 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.792816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b76f-0b9f-403a-b94d-8648966f6c39 None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/fd0db9d2a753490cb8473229a8f6fe15/roles/662ff20775c14ad69b5a84324701fcf1` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.794803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 480/950] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:51.795009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99b76f-0b9f-403a-b94d-8648966f6c39 None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=fd0db9d2a753490cb8473229a8f6fe15, role_id=662ff20775c14ad69b5a84324701fcf1)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:51.795634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e413e2-3e33-4409-945c-b5958ff38207 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:51.796445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99b76f-0b9f-403a-b94d-8648966f6c39 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:51.813199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 471/951] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:39:51 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/fd0db9d2a753490cb8473229a8f6fe15/roles/662ff20775c14ad69b5a84324701fcf1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:39:51.814556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd2dbc-c9a0-431f-94f6-ce16c24195dc None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.814844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd2dbc-c9a0-431f-94f6-ce16c24195dc None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.815113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd2dbc-c9a0-431f-94f6-ce16c24195dc None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.816469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 472/952] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:51.824303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19fe5-8b0a-4358-bdf8-c44882830108 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.824586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19fe5-8b0a-4358-bdf8-c44882830108 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.824859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19fe5-8b0a-4358-bdf8-c44882830108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.852862 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-81a19fe5-8b0a-4358-bdf8-c44882830108 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:51.855073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-81a19fe5-8b0a-4358-bdf8-c44882830108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:51.930558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e413e2-3e33-4409-945c-b5958ff38207 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:51.969935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 473/953] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:51.976804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50502f-2db6-4098-a2cc-4ff541a44021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:51.977555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50502f-2db6-4098-a2cc-4ff541a44021 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:51.977835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50502f-2db6-4098-a2cc-4ff541a44021 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:51.978107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50502f-2db6-4098-a2cc-4ff541a44021 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:51.978721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50502f-2db6-4098-a2cc-4ff541a44021 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:51.980242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50502f-2db6-4098-a2cc-4ff541a44021 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-3b50502f-2db6-4098-a2cc-4ff541a44021 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:51.990875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:51.993198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:51.995263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 481/954] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:51.999267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a19fe5-8b0a-4358-bdf8-c44882830108 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:52.003746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:52.106873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a19fe5-8b0a-4358-bdf8-c44882830108 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:52.142819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0810f9-48ce-412f-8eba-565b702bf6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:52.143430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0810f9-48ce-412f-8eba-565b702bf6c3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:52.143706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0810f9-48ce-412f-8eba-565b702bf6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:52.143980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0810f9-48ce-412f-8eba-565b702bf6c3 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:52.144675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0810f9-48ce-412f-8eba-565b702bf6c3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:52.144982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0810f9-48ce-412f-8eba-565b702bf6c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:52.146909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0810f9-48ce-412f-8eba-565b702bf6c3 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:52.152072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 474/955] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 3 14:39:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:52.157688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50502f-2db6-4098-a2cc-4ff541a44021 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:52.219746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:52.260147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0810f9-48ce-412f-8eba-565b702bf6c3 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:52.267487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50502f-2db6-4098-a2cc-4ff541a44021 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:52.304767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f36b0-7801-44c0-a09d-b50244e2b076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:52.305380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f36b0-7801-44c0-a09d-b50244e2b076 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:52.305695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f36b0-7801-44c0-a09d-b50244e2b076 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:52.305962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f36b0-7801-44c0-a09d-b50244e2b076 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:52.306592 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f36b0-7801-44c0-a09d-b50244e2b076 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:52.306891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f36b0-7801-44c0-a09d-b50244e2b076 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:52.308258 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f36b0-7801-44c0-a09d-b50244e2b076 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:52.317538 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 482/956] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 3 14:39:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:52.322490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f36b0-7801-44c0-a09d-b50244e2b076 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:52.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:52.396454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0810f9-48ce-412f-8eba-565b702bf6c3 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:52.430970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f36b0-7801-44c0-a09d-b50244e2b076 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:52.443484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ee92d-020a-4667-9e7d-46ba8df61523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:52.444360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ee92d-020a-4667-9e7d-46ba8df61523 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:52.444934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ee92d-020a-4667-9e7d-46ba8df61523 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:52.445367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ee92d-020a-4667-9e7d-46ba8df61523 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:52.446480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ee92d-020a-4667-9e7d-46ba8df61523 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:52.448256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ee92d-020a-4667-9e7d-46ba8df61523 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:52.449580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-c25ee92d-020a-4667-9e7d-46ba8df61523 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:52.459409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:52.460522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 14:39:52.461356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:52.467322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 475/957] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:52.468836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ee92d-020a-4667-9e7d-46ba8df61523 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:52.476363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4751849-703f-4878-9cb9-b740c2a8e5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:52.477008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4751849-703f-4878-9cb9-b740c2a8e5da None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:52.477296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4751849-703f-4878-9cb9-b740c2a8e5da None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:52.477644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4751849-703f-4878-9cb9-b740c2a8e5da None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:52.478327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4751849-703f-4878-9cb9-b740c2a8e5da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:52.479740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4751849-703f-4878-9cb9-b740c2a8e5da None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:52.509876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 483/958] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:39:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:39:52.587142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ee92d-020a-4667-9e7d-46ba8df61523 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:52.621694 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:52.640185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77ee92-1289-476a-947f-89cdbc6ba532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:52.641186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ee92-1289-476a-947f-89cdbc6ba532 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:52.641751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ee92-1289-476a-947f-89cdbc6ba532 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:52.642241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ee92-1289-476a-947f-89cdbc6ba532 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:52.643440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77ee92-1289-476a-947f-89cdbc6ba532 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:52.644058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77ee92-1289-476a-947f-89cdbc6ba532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:52.646278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77ee92-1289-476a-947f-89cdbc6ba532 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:52.652548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 476/959] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:52.658715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4751849-703f-4878-9cb9-b740c2a8e5da None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:52.771974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4751849-703f-4878-9cb9-b740c2a8e5da None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:52.807101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2af6b0-406d-4850-ad92-c9a5b9176c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:52.807707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2af6b0-406d-4850-ad92-c9a5b9176c05 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:52.807982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2af6b0-406d-4850-ad92-c9a5b9176c05 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:52.808251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2af6b0-406d-4850-ad92-c9a5b9176c05 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:52.808875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2af6b0-406d-4850-ad92-c9a5b9176c05 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:52.809173 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2af6b0-406d-4850-ad92-c9a5b9176c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:52.810567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2af6b0-406d-4850-ad92-c9a5b9176c05 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:52.818227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 484/960] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:54.018627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0b70a-2d8a-42cf-94e2-b0a25878cde5 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.019197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0b70a-2d8a-42cf-94e2-b0a25878cde5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.019197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0b70a-2d8a-42cf-94e2-b0a25878cde5 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.020920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 477/961] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:54.026975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857591d5-2274-4e71-97ee-11d51dd4249e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.027491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857591d5-2274-4e71-97ee-11d51dd4249e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.027491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857591d5-2274-4e71-97ee-11d51dd4249e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.048682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-857591d5-2274-4e71-97ee-11d51dd4249e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:54.049964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-857591d5-2274-4e71-97ee-11d51dd4249e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:54.085239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:54.125377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 485/962] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:54.149052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30110add-0c19-4685-b476-947a6fefa881 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.149337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30110add-0c19-4685-b476-947a6fefa881 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.149654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30110add-0c19-4685-b476-947a6fefa881 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.151241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 478/963] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:54.156381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457fd4a-fada-410c-bf7c-3aecd003803d None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.156671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457fd4a-fada-410c-bf7c-3aecd003803d None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.156944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457fd4a-fada-410c-bf7c-3aecd003803d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.174875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-8457fd4a-fada-410c-bf7c-3aecd003803d None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:54.176061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8457fd4a-fada-410c-bf7c-3aecd003803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:54.244523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 486/964] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:54.392254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30110add-0c19-4685-b476-947a6fefa881 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:54.501555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30110add-0c19-4685-b476-947a6fefa881 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:54.518759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457fd4a-fada-410c-bf7c-3aecd003803d None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:54.531932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745e4f1-5071-4ee3-9536-3d10f7eb118e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:54.532736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745e4f1-5071-4ee3-9536-3d10f7eb118e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.533020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745e4f1-5071-4ee3-9536-3d10f7eb118e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.533291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745e4f1-5071-4ee3-9536-3d10f7eb118e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.550097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745e4f1-5071-4ee3-9536-3d10f7eb118e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-e745e4f1-5071-4ee3-9536-3d10f7eb118e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:39:54.557449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:54.558580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:39:54.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:54.561935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 479/965] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 3 14:39:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:54.569236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745e4f1-5071-4ee3-9536-3d10f7eb118e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:54.624374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:54.639233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:54.643159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457fd4a-fada-410c-bf7c-3aecd003803d None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:54.681059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c474bc6-a5f7-4401-9d30-25d78f488f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:54.682148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c474bc6-a5f7-4401-9d30-25d78f488f00 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.682639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c474bc6-a5f7-4401-9d30-25d78f488f00 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.683116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c474bc6-a5f7-4401-9d30-25d78f488f00 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.684025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c474bc6-a5f7-4401-9d30-25d78f488f00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:54.685776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c474bc6-a5f7-4401-9d30-25d78f488f00 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:54.691680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745e4f1-5071-4ee3-9536-3d10f7eb118e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:54.707611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.sql.core [None req-1c474bc6-a5f7-4401-9d30-25d78f488f00 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 14:39:54.707611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 03 14:39:54.707611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [parameters: {'user_id': '8b77f31fd64c45a8a6818c2f0696ef79', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 03 14:39:54.707611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87831) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-1c474bc6-a5f7-4401-9d30-25d78f488f00 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application conn.query(q) Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application result.read() Oct 03 14:39:54.716899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 14:39:54.717786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application next(self.gen) Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application yield resource Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application next(self.gen) Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise self.value Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application session.commit() Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 14:39:54.718735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self.session.flush() Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application compat.raise_( Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 14:39:54.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 14:39:54.720565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application conn.query(q) Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application result.read() Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 14:39:54.721425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [parameters: {'user_id': '8b77f31fd64c45a8a6818c2f0696ef79', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:54.724938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 03 14:39:54.726939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 03 14:39:54.729078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 03 14:39:54.729078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 03 14:39:54.729078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 14:39:54.729078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 14:39:54.729078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 03 14:39:54.729078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:54.729078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 487/966] 10.222.0.46 () {60 vars in 1131 bytes} [Mon Oct 3 14:39:54 2022] POST /identity/v3/users => generated 160 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:54.729078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c474bc6-a5f7-4401-9d30-25d78f488f00 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:54.745389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1784c-ba4a-4b38-bc11-4d85d1a1a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:54.746764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1784c-ba4a-4b38-bc11-4d85d1a1a205 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.747499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1784c-ba4a-4b38-bc11-4d85d1a1a205 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.747970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1784c-ba4a-4b38-bc11-4d85d1a1a205 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.748925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1784c-ba4a-4b38-bc11-4d85d1a1a205 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:54.749529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1784c-ba4a-4b38-bc11-4d85d1a1a205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:54.751174 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1784c-ba4a-4b38-bc11-4d85d1a1a205 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:54.766863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 480/967] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 3 14:39:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:54.768498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1784c-ba4a-4b38-bc11-4d85d1a1a205 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:54.850034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c474bc6-a5f7-4401-9d30-25d78f488f00 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:54.886372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1784c-ba4a-4b38-bc11-4d85d1a1a205 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:54.890119 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cb2ff-2a94-4e73-9659-0a73a612be07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:54.890755 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cb2ff-2a94-4e73-9659-0a73a612be07 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.891037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cb2ff-2a94-4e73-9659-0a73a612be07 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.891312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cb2ff-2a94-4e73-9659-0a73a612be07 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.899215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384cb2ff-2a94-4e73-9659-0a73a612be07 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:54.900855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384cb2ff-2a94-4e73-9659-0a73a612be07 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:54.901750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-384cb2ff-2a94-4e73-9659-0a73a612be07 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:54.913436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:54.914494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:54.915446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 488/968] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:39:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:54.916060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e48452-f71f-4ea6-b8c0-e883ef47b8ad None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.916060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e48452-f71f-4ea6-b8c0-e883ef47b8ad None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.916822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e48452-f71f-4ea6-b8c0-e883ef47b8ad None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.920702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 489/969] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:54.922280 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e48452-f71f-4ea6-b8c0-e883ef47b8ad None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:54.931109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643316f-e900-489c-8905-7ce7be95f290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:54.931754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643316f-e900-489c-8905-7ce7be95f290 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.932031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643316f-e900-489c-8905-7ce7be95f290 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.932300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643316f-e900-489c-8905-7ce7be95f290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.945270 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643316f-e900-489c-8905-7ce7be95f290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:54.946799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643316f-e900-489c-8905-7ce7be95f290 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:54.947869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-f643316f-e900-489c-8905-7ce7be95f290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:54.965578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:54.966819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:54.967753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:54.967753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:54.967753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:54.967753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 481/970] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:54.972730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7bb5-a724-4539-9fd5-f0a61bebac24 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:54.973017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7bb5-a724-4539-9fd5-f0a61bebac24 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:54.973292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f7bb5-a724-4539-9fd5-f0a61bebac24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:54.992427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a88f7bb5-a724-4539-9fd5-f0a61bebac24 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:54.993632 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a88f7bb5-a724-4539-9fd5-f0a61bebac24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:55.042285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e48452-f71f-4ea6-b8c0-e883ef47b8ad None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:55.064325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 482/971] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:55.065384 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f7bb5-a724-4539-9fd5-f0a61bebac24 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:55.080714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f049b15-c063-40e4-88f3-e45f141a9192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:55.081620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f049b15-c063-40e4-88f3-e45f141a9192 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.081902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f049b15-c063-40e4-88f3-e45f141a9192 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.082164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f049b15-c063-40e4-88f3-e45f141a9192 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.083550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f049b15-c063-40e4-88f3-e45f141a9192 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:55.084147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f049b15-c063-40e4-88f3-e45f141a9192 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:55.085615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f049b15-c063-40e4-88f3-e45f141a9192 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:55.123858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 490/972] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 3 14:39:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:55.125010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff6e59-949e-4da7-86d0-835ebcb2c2d6 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.125286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff6e59-949e-4da7-86d0-835ebcb2c2d6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.125583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff6e59-949e-4da7-86d0-835ebcb2c2d6 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.126964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 491/973] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:55.132673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2d01c-6320-4a97-b553-16eb64819008 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.132986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2d01c-6320-4a97-b553-16eb64819008 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.133220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2d01c-6320-4a97-b553-16eb64819008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.153344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d7c2d01c-6320-4a97-b553-16eb64819008 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:55.155077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d7c2d01c-6320-4a97-b553-16eb64819008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:55.175961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:55.183256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f7bb5-a724-4539-9fd5-f0a61bebac24 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:55.228921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e5a73-e5af-4e9f-8e47-a3e4c918136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:55.229629 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e5a73-e5af-4e9f-8e47-a3e4c918136c None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.229899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e5a73-e5af-4e9f-8e47-a3e4c918136c None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.230165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e5a73-e5af-4e9f-8e47-a3e4c918136c None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.231267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e5a73-e5af-4e9f-8e47-a3e4c918136c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:55.231571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e5a73-e5af-4e9f-8e47-a3e4c918136c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:55.233284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e5a73-e5af-4e9f-8e47-a3e4c918136c None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:55.240054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 492/974] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:55.256834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 483/975] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:55.258307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2d01c-6320-4a97-b553-16eb64819008 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:55.270078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:55.351979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e5a73-e5af-4e9f-8e47-a3e4c918136c None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:55.380218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2d01c-6320-4a97-b553-16eb64819008 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:55.422715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05eb50-1f29-4dc7-ad5a-38034620770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:55.423354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05eb50-1f29-4dc7-ad5a-38034620770e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.423634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05eb50-1f29-4dc7-ad5a-38034620770e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.423904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05eb50-1f29-4dc7-ad5a-38034620770e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.432235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05eb50-1f29-4dc7-ad5a-38034620770e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:55.433710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05eb50-1f29-4dc7-ad5a-38034620770e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-7e05eb50-1f29-4dc7-ad5a-38034620770e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:39:55.447731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:55.448707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:55.449683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 493/976] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 3 14:39:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:55.454505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05eb50-1f29-4dc7-ad5a-38034620770e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:55.484980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e5a73-e5af-4e9f-8e47-a3e4c918136c None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:55.521559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ead3b-b2a1-45cd-8fcf-4fca64a8daac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:55.522347 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ead3b-b2a1-45cd-8fcf-4fca64a8daac None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.522606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ead3b-b2a1-45cd-8fcf-4fca64a8daac None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.522879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ead3b-b2a1-45cd-8fcf-4fca64a8daac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.523466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ead3b-b2a1-45cd-8fcf-4fca64a8daac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:55.524828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ead3b-b2a1-45cd-8fcf-4fca64a8daac None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:55.528749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-3d0ead3b-b2a1-45cd-8fcf-4fca64a8daac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:55.534272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:55.535259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:55.536168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:55.537863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 484/977] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:55.538130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ead3b-b2a1-45cd-8fcf-4fca64a8daac None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:55.560590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:55.561742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05eb50-1f29-4dc7-ad5a-38034620770e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:55.595089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e4632-0001-4eac-bcbd-e9b79aafa997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:55.595688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e4632-0001-4eac-bcbd-e9b79aafa997 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.595972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e4632-0001-4eac-bcbd-e9b79aafa997 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.596242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e4632-0001-4eac-bcbd-e9b79aafa997 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.596958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e4632-0001-4eac-bcbd-e9b79aafa997 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:55.597262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e4632-0001-4eac-bcbd-e9b79aafa997 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:55.598722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e4632-0001-4eac-bcbd-e9b79aafa997 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:55.616114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 494/978] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 3 14:39:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:55.617184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e4632-0001-4eac-bcbd-e9b79aafa997 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:55.639241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ead3b-b2a1-45cd-8fcf-4fca64a8daac None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:55.670834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cb4d7-3f48-4eb6-b079-80713388f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:55.671425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cb4d7-3f48-4eb6-b079-80713388f393 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.671701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cb4d7-3f48-4eb6-b079-80713388f393 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.671968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cb4d7-3f48-4eb6-b079-80713388f393 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.672609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cb4d7-3f48-4eb6-b079-80713388f393 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:55.673992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cb4d7-3f48-4eb6-b079-80713388f393 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:55.688287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 485/979] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:39:55 2022] POST /identity/v3/services => generated 242 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:39:55.689080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cb4d7-3f48-4eb6-b079-80713388f393 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:55.725697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e4632-0001-4eac-bcbd-e9b79aafa997 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:55.755586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:55.773908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b41ebe-3643-4c8a-9065-facb07b5e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:55.774530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b41ebe-3643-4c8a-9065-facb07b5e939 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.774981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b41ebe-3643-4c8a-9065-facb07b5e939 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.775150 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b41ebe-3643-4c8a-9065-facb07b5e939 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.775884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b41ebe-3643-4c8a-9065-facb07b5e939 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:55.776240 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b41ebe-3643-4c8a-9065-facb07b5e939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:55.777740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b41ebe-3643-4c8a-9065-facb07b5e939 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:55.781333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:55.788192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 495/980] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 3 14:39:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:55.789704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b41ebe-3643-4c8a-9065-facb07b5e939 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:55.805547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cb4d7-3f48-4eb6-b079-80713388f393 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:55.846101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e0687-69ec-4333-bbcf-3bc32c4d529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:55.846883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e0687-69ec-4333-bbcf-3bc32c4d529e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.847161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e0687-69ec-4333-bbcf-3bc32c4d529e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.847474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e0687-69ec-4333-bbcf-3bc32c4d529e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.902302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b41ebe-3643-4c8a-9065-facb07b5e939 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:55.934034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e0687-69ec-4333-bbcf-3bc32c4d529e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:55.935499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e0687-69ec-4333-bbcf-3bc32c4d529e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:55.936924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1409a51-a0e9-4d1c-8896-b952b117993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:55.937690 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1409a51-a0e9-4d1c-8896-b952b117993d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:55.938013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1409a51-a0e9-4d1c-8896-b952b117993d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:55.938286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1409a51-a0e9-4d1c-8896-b952b117993d None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:55.938917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1409a51-a0e9-4d1c-8896-b952b117993d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:55.939222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1409a51-a0e9-4d1c-8896-b952b117993d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:55.940573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1409a51-a0e9-4d1c-8896-b952b117993d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:55.950808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 496/981] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 3 14:39:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:55.952446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1409a51-a0e9-4d1c-8896-b952b117993d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:56.007168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:56.028500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:56.053721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 486/982] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:39:55 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:39:56.058391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e0687-69ec-4333-bbcf-3bc32c4d529e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:56.063811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1409a51-a0e9-4d1c-8896-b952b117993d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:56.104314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af03a07-8a29-4e9e-88c0-d2727c6d1e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:56.105051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af03a07-8a29-4e9e-88c0-d2727c6d1e22 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:56.105350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af03a07-8a29-4e9e-88c0-d2727c6d1e22 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:56.105869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af03a07-8a29-4e9e-88c0-d2727c6d1e22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:56.106603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af03a07-8a29-4e9e-88c0-d2727c6d1e22 None admin] RBAC: Adding query filter params (interface=admin, service_id=607f0da4bf9b46fa8e9b18c8a0532fff, region_id=RegionOne) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:56.106919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af03a07-8a29-4e9e-88c0-d2727c6d1e22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:56.108312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af03a07-8a29-4e9e-88c0-d2727c6d1e22 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:56.118239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 497/983] 10.222.0.46 () {56 vars in 1249 bytes} [Mon Oct 3 14:39:55 2022] GET /identity/v3/endpoints?service_id=607f0da4bf9b46fa8e9b18c8a0532fff&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:56.119760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af03a07-8a29-4e9e-88c0-d2727c6d1e22 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:56.163608 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e0687-69ec-4333-bbcf-3bc32c4d529e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:56.200015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c685907-68c8-426d-b935-703dbd69e93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:56.201050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c685907-68c8-426d-b935-703dbd69e93d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:56.201332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c685907-68c8-426d-b935-703dbd69e93d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:56.201635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c685907-68c8-426d-b935-703dbd69e93d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:56.212390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c685907-68c8-426d-b935-703dbd69e93d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:56.213859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c685907-68c8-426d-b935-703dbd69e93d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:56.227377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af03a07-8a29-4e9e-88c0-d2727c6d1e22 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-3c685907-68c8-426d-b935-703dbd69e93d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:39:56.232813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:56.233738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:39:56.235225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:39:56.235225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:39:56.235225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:56.235225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 487/984] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:56.239680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c685907-68c8-426d-b935-703dbd69e93d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:56.244006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:56.266426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da995f-14ea-4981-8ec9-d7711dba8fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:56.267274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da995f-14ea-4981-8ec9-d7711dba8fee None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:56.267553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da995f-14ea-4981-8ec9-d7711dba8fee None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:56.267822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da995f-14ea-4981-8ec9-d7711dba8fee None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:56.268417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da995f-14ea-4981-8ec9-d7711dba8fee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:56.269794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da995f-14ea-4981-8ec9-d7711dba8fee None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:56.274934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:56.276356 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 498/985] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 3 14:39:56 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:39:56.363914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c685907-68c8-426d-b935-703dbd69e93d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:56.401517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7adc8-5292-4a69-8460-c9c0be6b60aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:56.402189 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7adc8-5292-4a69-8460-c9c0be6b60aa None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:56.402469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7adc8-5292-4a69-8460-c9c0be6b60aa None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:56.402741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7adc8-5292-4a69-8460-c9c0be6b60aa None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:56.403408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7adc8-5292-4a69-8460-c9c0be6b60aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:56.403719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7adc8-5292-4a69-8460-c9c0be6b60aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:56.405125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7adc8-5292-4a69-8460-c9c0be6b60aa None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:56.423250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 488/986] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:56.424482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da995f-14ea-4981-8ec9-d7711dba8fee None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:56.535695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da995f-14ea-4981-8ec9-d7711dba8fee None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:56.570596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc487ffd-fe0d-42e6-a32a-8faf1a3f2847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:56.571481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc487ffd-fe0d-42e6-a32a-8faf1a3f2847 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:56.571916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc487ffd-fe0d-42e6-a32a-8faf1a3f2847 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:56.572332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc487ffd-fe0d-42e6-a32a-8faf1a3f2847 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:56.579556 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc487ffd-fe0d-42e6-a32a-8faf1a3f2847 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:56.581294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc487ffd-fe0d-42e6-a32a-8faf1a3f2847 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-cc487ffd-fe0d-42e6-a32a-8faf1a3f2847 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:39:56.595142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:56.596175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:56.597091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:39:56.597091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:39:56.597091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:39:56.597091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:39:56.597091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:39:56.597091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:39:56.597091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:39:56.597091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:39:56.597580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 499/987] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:56.603681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7adc8-5292-4a69-8460-c9c0be6b60aa None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:56.697044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:56.706980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7adc8-5292-4a69-8460-c9c0be6b60aa None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:56.741380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f758b3-9811-4e1b-a718-34ab49b45b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:56.742020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f758b3-9811-4e1b-a718-34ab49b45b7d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:56.742302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f758b3-9811-4e1b-a718-34ab49b45b7d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:56.742572 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f758b3-9811-4e1b-a718-34ab49b45b7d None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:56.743263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f758b3-9811-4e1b-a718-34ab49b45b7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:56.743569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f758b3-9811-4e1b-a718-34ab49b45b7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:56.745021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f758b3-9811-4e1b-a718-34ab49b45b7d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:56.756891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 489/988] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:39:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:56.762669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc487ffd-fe0d-42e6-a32a-8faf1a3f2847 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:56.804888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:56.867813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc487ffd-fe0d-42e6-a32a-8faf1a3f2847 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:56.904943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204815c0-73a7-4fbf-804d-a87e024782ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:56.905588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204815c0-73a7-4fbf-804d-a87e024782ad None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:56.905867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204815c0-73a7-4fbf-804d-a87e024782ad None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:56.906134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204815c0-73a7-4fbf-804d-a87e024782ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:56.906858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204815c0-73a7-4fbf-804d-a87e024782ad None admin] RBAC: Adding query filter params (role.id=662ff20775c14ad69b5a84324701fcf1, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=fd0db9d2a753490cb8473229a8f6fe15) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:56.907160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204815c0-73a7-4fbf-804d-a87e024782ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:56.908570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204815c0-73a7-4fbf-804d-a87e024782ad None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:56.921591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 500/989] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:39:56 2022] GET /identity/v3/role_assignments?user.id=fd0db9d2a753490cb8473229a8f6fe15&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=662ff20775c14ad69b5a84324701fcf1 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:57.865975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9638d18-7147-4609-9ab4-67ae7a776d52 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:57.866524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9638d18-7147-4609-9ab4-67ae7a776d52 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:57.866524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9638d18-7147-4609-9ab4-67ae7a776d52 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:57.867994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 490/990] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:39:57.872901 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716640f5-3e0f-4d67-8ebb-bd8876515eeb None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:57.873273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716640f5-3e0f-4d67-8ebb-bd8876515eeb None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:57.873460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716640f5-3e0f-4d67-8ebb-bd8876515eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:57.891162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-716640f5-3e0f-4d67-8ebb-bd8876515eeb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:57.892335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-716640f5-3e0f-4d67-8ebb-bd8876515eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:57.959868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 501/991] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:39:58.223984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9638d18-7147-4609-9ab4-67ae7a776d52 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:58.338903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9638d18-7147-4609-9ab4-67ae7a776d52 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:58.372861 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c1e06-1a6d-44ef-bbfc-70390c80ea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:58.373491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c1e06-1a6d-44ef-bbfc-70390c80ea6f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:58.373792 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c1e06-1a6d-44ef-bbfc-70390c80ea6f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:58.374065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c1e06-1a6d-44ef-bbfc-70390c80ea6f None admin] PATH_INFO: `/v3/services/compute` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:58.374642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c1e06-1a6d-44ef-bbfc-70390c80ea6f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:58.376011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c1e06-1a6d-44ef-bbfc-70390c80ea6f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-d67c1e06-1a6d-44ef-bbfc-70390c80ea6f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:58.384775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:58.386006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 14:39:58.386887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:58.389826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 491/992] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:39:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:39:58.391718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716640f5-3e0f-4d67-8ebb-bd8876515eeb None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:58.416274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d218b55-ca0e-4071-a0b9-c45dc0868888 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:58.416565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d218b55-ca0e-4071-a0b9-c45dc0868888 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:58.416844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d218b55-ca0e-4071-a0b9-c45dc0868888 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:58.418309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 492/993] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:58.423060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d0ec1-253f-4f76-951e-9d69a43e6d8c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:58.423339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d0ec1-253f-4f76-951e-9d69a43e6d8c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:58.423604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d0ec1-253f-4f76-951e-9d69a43e6d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:58.424393 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:58.441596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-778d0ec1-253f-4f76-951e-9d69a43e6d8c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:58.442803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-778d0ec1-253f-4f76-951e-9d69a43e6d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:58.454012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:58.500365 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716640f5-3e0f-4d67-8ebb-bd8876515eeb None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:58.513978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 493/994] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:58.540699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482ee94-c9b3-4baf-a8a4-8136f2a57485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:58.541305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482ee94-c9b3-4baf-a8a4-8136f2a57485 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:58.541658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482ee94-c9b3-4baf-a8a4-8136f2a57485 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:58.541929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482ee94-c9b3-4baf-a8a4-8136f2a57485 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:58.542553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482ee94-c9b3-4baf-a8a4-8136f2a57485 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:58.542852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482ee94-c9b3-4baf-a8a4-8136f2a57485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:58.544206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482ee94-c9b3-4baf-a8a4-8136f2a57485 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:58.554355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 502/995] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:58.555173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d0ec1-253f-4f76-951e-9d69a43e6d8c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:58.653254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d0ec1-253f-4f76-951e-9d69a43e6d8c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:58.687212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33bf66-dbac-4537-ae36-814b30d8dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:58.687813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33bf66-dbac-4537-ae36-814b30d8dfda None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:58.688091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33bf66-dbac-4537-ae36-814b30d8dfda None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:58.688374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33bf66-dbac-4537-ae36-814b30d8dfda None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:58.688984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33bf66-dbac-4537-ae36-814b30d8dfda None admin] RBAC: Adding query filter params (type=compute) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:58.689283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33bf66-dbac-4537-ae36-814b30d8dfda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:58.690679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33bf66-dbac-4537-ae36-814b30d8dfda None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:58.700851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 494/996] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:39:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:58.701982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482ee94-c9b3-4baf-a8a4-8136f2a57485 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:58.796115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:58.803632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482ee94-c9b3-4baf-a8a4-8136f2a57485 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:58.812730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33bf66-dbac-4537-ae36-814b30d8dfda None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:58.825560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:58.840069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c4f2e-930b-430e-9462-eef49e3d97ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:58.840684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c4f2e-930b-430e-9462-eef49e3d97ad None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:58.840961 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c4f2e-930b-430e-9462-eef49e3d97ad None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:58.841252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c4f2e-930b-430e-9462-eef49e3d97ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:58.841960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c4f2e-930b-430e-9462-eef49e3d97ad None admin] RBAC: Adding query filter params (interface=public, service_id=b4034f4a73574a04bf5dab11f8226ec9, region_id=RegionOne) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:58.842268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c4f2e-930b-430e-9462-eef49e3d97ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:58.843658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c4f2e-930b-430e-9462-eef49e3d97ad None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:58.849055 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 503/997] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 3 14:39:58 2022] GET /identity/v3/endpoints?service_id=b4034f4a73574a04bf5dab11f8226ec9&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:58.854387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c4f2e-930b-430e-9462-eef49e3d97ad None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:58.927145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33bf66-dbac-4537-ae36-814b30d8dfda None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:58.965276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a26f1-183a-4f5d-a8b6-0f014610f82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:58.966084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a26f1-183a-4f5d-a8b6-0f014610f82b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:58.966363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a26f1-183a-4f5d-a8b6-0f014610f82b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:58.966633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a26f1-183a-4f5d-a8b6-0f014610f82b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:58.967220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a26f1-183a-4f5d-a8b6-0f014610f82b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:58.968577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a26f1-183a-4f5d-a8b6-0f014610f82b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:58.970650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c4f2e-930b-430e-9462-eef49e3d97ad None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-4e2a26f1-183a-4f5d-a8b6-0f014610f82b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:58.977795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:58.978889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:58.979743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 495/998] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 3 14:39:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:58.985689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a26f1-183a-4f5d-a8b6-0f014610f82b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:59.013118 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd70307-f2a5-400f-93ff-9d32bf4b34c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:59.013771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd70307-f2a5-400f-93ff-9d32bf4b34c2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:59.014054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd70307-f2a5-400f-93ff-9d32bf4b34c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:59.014318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd70307-f2a5-400f-93ff-9d32bf4b34c2 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:59.014993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd70307-f2a5-400f-93ff-9d32bf4b34c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:59.016413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd70307-f2a5-400f-93ff-9d32bf4b34c2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:59.033528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 504/999] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 3 14:39:58 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:39:59.080302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:59.104995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f589b8-030c-4827-9668-c36f02683502 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:59.105322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f589b8-030c-4827-9668-c36f02683502 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:59.105593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f589b8-030c-4827-9668-c36f02683502 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:59.107159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 505/1000] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:39:59.111287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a26f1-183a-4f5d-a8b6-0f014610f82b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:59.112534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5e803-0347-4769-895d-983c9b3a7f39 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:59.112825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5e803-0347-4769-895d-983c9b3a7f39 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:59.113362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5e803-0347-4769-895d-983c9b3a7f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:59.132314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-bef5e803-0347-4769-895d-983c9b3a7f39 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:39:59.133598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bef5e803-0347-4769-895d-983c9b3a7f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:39:59.150857 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580da71-1998-437b-8790-c2b12d4dff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:59.151475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580da71-1998-437b-8790-c2b12d4dff79 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:59.151756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580da71-1998-437b-8790-c2b12d4dff79 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:59.152029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580da71-1998-437b-8790-c2b12d4dff79 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:59.152678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580da71-1998-437b-8790-c2b12d4dff79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:59.152986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580da71-1998-437b-8790-c2b12d4dff79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:59.153302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:59.154633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580da71-1998-437b-8790-c2b12d4dff79 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:59.165296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 496/1001] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 3 14:39:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:59.166973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580da71-1998-437b-8790-c2b12d4dff79 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:59.216663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 506/1002] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:39:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:39:59.276727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580da71-1998-437b-8790-c2b12d4dff79 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:59.315405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58267e7-4089-42a8-a829-ba32963041d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:59.316013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58267e7-4089-42a8-a829-ba32963041d1 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:59.316291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58267e7-4089-42a8-a829-ba32963041d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:59.316560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58267e7-4089-42a8-a829-ba32963041d1 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:59.317214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58267e7-4089-42a8-a829-ba32963041d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:59.317540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58267e7-4089-42a8-a829-ba32963041d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:59.318999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58267e7-4089-42a8-a829-ba32963041d1 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:59.328338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 497/1003] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 3 14:39:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:39:59.329894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5e803-0347-4769-895d-983c9b3a7f39 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:59.430573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5e803-0347-4769-895d-983c9b3a7f39 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:59.463976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140f546-3715-4c52-8943-f28154d9db57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:59.464577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140f546-3715-4c52-8943-f28154d9db57 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:59.464848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140f546-3715-4c52-8943-f28154d9db57 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:59.465110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140f546-3715-4c52-8943-f28154d9db57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:59.465807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140f546-3715-4c52-8943-f28154d9db57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:59.467167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140f546-3715-4c52-8943-f28154d9db57 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:59.484143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58267e7-4089-42a8-a829-ba32963041d1 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:59.485031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 507/1004] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:39:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:39:59.495995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:59.605373 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58267e7-4089-42a8-a829-ba32963041d1 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:59.648271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c8b6f-84c0-4183-9c30-4cf3d983b437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:59.648924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c8b6f-84c0-4183-9c30-4cf3d983b437 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:59.649205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c8b6f-84c0-4183-9c30-4cf3d983b437 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:59.649478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c8b6f-84c0-4183-9c30-4cf3d983b437 None admin] PATH_INFO: `/v3/services/image` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:59.650102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c8b6f-84c0-4183-9c30-4cf3d983b437 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:59.651553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c8b6f-84c0-4183-9c30-4cf3d983b437 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-ab4c8b6f-84c0-4183-9c30-4cf3d983b437 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:39:59.661660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:39:59.662642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:39:59.663475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 498/1005] 10.222.0.46 () {56 vars in 1105 bytes} [Mon Oct 3 14:39:59 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:39:59.669064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140f546-3715-4c52-8943-f28154d9db57 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:59.685342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:59.774496 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140f546-3715-4c52-8943-f28154d9db57 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:59.808139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7aebc-c863-4a50-b8c4-d9d0f4421e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:59.808755 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7aebc-c863-4a50-b8c4-d9d0f4421e64 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:59.809034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7aebc-c863-4a50-b8c4-d9d0f4421e64 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:59.809303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7aebc-c863-4a50-b8c4-d9d0f4421e64 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:59.809960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7aebc-c863-4a50-b8c4-d9d0f4421e64 None admin] RBAC: Adding query filter params (name=image) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:59.810263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7aebc-c863-4a50-b8c4-d9d0f4421e64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:59.811627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7aebc-c863-4a50-b8c4-d9d0f4421e64 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:59.816676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 508/1006] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:39:59.823005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c8b6f-84c0-4183-9c30-4cf3d983b437 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:39:59.885391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:39:59.926047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c8b6f-84c0-4183-9c30-4cf3d983b437 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:39:59.961139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cccb27-66ba-4aed-889b-b69f40d9aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:39:59.961837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cccb27-66ba-4aed-889b-b69f40d9aadb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:39:59.962089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cccb27-66ba-4aed-889b-b69f40d9aadb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:39:59.962257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cccb27-66ba-4aed-889b-b69f40d9aadb None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:39:59.962852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cccb27-66ba-4aed-889b-b69f40d9aadb None admin] RBAC: Adding query filter params (type=image) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:39:59.963155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cccb27-66ba-4aed-889b-b69f40d9aadb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:39:59.964515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cccb27-66ba-4aed-889b-b69f40d9aadb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:39:59.977526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 499/1007] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:39:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:01.215010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c124e-24e3-4f0f-880c-df8062bc1dc9 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:01.215539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c124e-24e3-4f0f-880c-df8062bc1dc9 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:01.215539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c124e-24e3-4f0f-880c-df8062bc1dc9 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:01.217030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 509/1008] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:01.222070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1dd84-1304-4a35-82ff-e8dfe202de02 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:01.222458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1dd84-1304-4a35-82ff-e8dfe202de02 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:01.222618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1dd84-1304-4a35-82ff-e8dfe202de02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:01.241668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-97e1dd84-1304-4a35-82ff-e8dfe202de02 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:01.242856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-97e1dd84-1304-4a35-82ff-e8dfe202de02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:01.315865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 500/1009] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:01 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:01.583903 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c124e-24e3-4f0f-880c-df8062bc1dc9 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:01.670179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:01.694894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c124e-24e3-4f0f-880c-df8062bc1dc9 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:01.726546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5f3f7-96fb-4dbf-a627-0132224a4cf2 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:01.726832 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5f3f7-96fb-4dbf-a627-0132224a4cf2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:01.727104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5f3f7-96fb-4dbf-a627-0132224a4cf2 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:01.728549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 501/1010] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:40:01.728907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7909373-7ed8-4863-92c2-9792703f9392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:01.729561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7909373-7ed8-4863-92c2-9792703f9392 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:01.729844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7909373-7ed8-4863-92c2-9792703f9392 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:01.730120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7909373-7ed8-4863-92c2-9792703f9392 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:01.730730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7909373-7ed8-4863-92c2-9792703f9392 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:01.732094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7909373-7ed8-4863-92c2-9792703f9392 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:01.733539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb548-9b84-4316-aa42-7882eacb1a4c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:01.733829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb548-9b84-4316-aa42-7882eacb1a4c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:01.734103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb548-9b84-4316-aa42-7882eacb1a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-b7909373-7ed8-4863-92c2-9792703f9392 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:01.742162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:01.743159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:01.744361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 510/1011] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:40:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:01.748897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7909373-7ed8-4863-92c2-9792703f9392 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:01.756725 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-48afb548-9b84-4316-aa42-7882eacb1a4c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:01.757998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-48afb548-9b84-4316-aa42-7882eacb1a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:01.812823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:01.834167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 502/1012] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:01 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:01.853475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7909373-7ed8-4863-92c2-9792703f9392 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:01.888634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0819e5e-b407-45a2-9d23-a2cc270d902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:01.889226 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0819e5e-b407-45a2-9d23-a2cc270d902f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:01.889590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0819e5e-b407-45a2-9d23-a2cc270d902f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:01.889790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0819e5e-b407-45a2-9d23-a2cc270d902f None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:01.890408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0819e5e-b407-45a2-9d23-a2cc270d902f None admin] RBAC: Adding query filter params (name=compute) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:01.890706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0819e5e-b407-45a2-9d23-a2cc270d902f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:01.892086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0819e5e-b407-45a2-9d23-a2cc270d902f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:01.901563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 511/1013] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:40:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:01.906085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afb548-9b84-4316-aa42-7882eacb1a4c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:02.016381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afb548-9b84-4316-aa42-7882eacb1a4c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:02.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a3ee6-57d6-4624-9d6c-a368f737b6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:02.059040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a3ee6-57d6-4624-9d6c-a368f737b6da None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:02.059314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a3ee6-57d6-4624-9d6c-a368f737b6da None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:02.059586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a3ee6-57d6-4624-9d6c-a368f737b6da None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:02.060209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a3ee6-57d6-4624-9d6c-a368f737b6da None admin] RBAC: Adding query filter params (type=compute) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:02.060506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a3ee6-57d6-4624-9d6c-a368f737b6da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:02.061890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a3ee6-57d6-4624-9d6c-a368f737b6da None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:02.071378 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 503/1014] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 3 14:40:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:02.072341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0819e5e-b407-45a2-9d23-a2cc270d902f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:02.081596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:02.105817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a3ee6-57d6-4624-9d6c-a368f737b6da None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:02.159917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:02.177585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0819e5e-b407-45a2-9d23-a2cc270d902f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:02.209334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a3ee6-57d6-4624-9d6c-a368f737b6da None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:02.210300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd4272-da48-44b4-beb1-3d6debbc884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:02.210906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd4272-da48-44b4-beb1-3d6debbc884d None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:02.211181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd4272-da48-44b4-beb1-3d6debbc884d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:02.211457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd4272-da48-44b4-beb1-3d6debbc884d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:02.212115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd4272-da48-44b4-beb1-3d6debbc884d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:02.213477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd4272-da48-44b4-beb1-3d6debbc884d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:02.232844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 512/1015] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:40:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:40:02.234208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeccb5f-164f-4dd4-b7ef-c40d4d04c1e6 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:02.234344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeccb5f-164f-4dd4-b7ef-c40d4d04c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:02.234586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeccb5f-164f-4dd4-b7ef-c40d4d04c1e6 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:02.235796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 513/1016] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:02.242411 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da75fc-e9f4-48ec-832e-bdf4f8509b24 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:02.242695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da75fc-e9f4-48ec-832e-bdf4f8509b24 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:02.242974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da75fc-e9f4-48ec-832e-bdf4f8509b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:02.261859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-31da75fc-e9f4-48ec-832e-bdf4f8509b24 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:02.262916 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67f9b0-a207-4bb5-87cb-06c1b3ebd077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:02.263380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-31da75fc-e9f4-48ec-832e-bdf4f8509b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:02.264183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67f9b0-a207-4bb5-87cb-06c1b3ebd077 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:02.264492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67f9b0-a207-4bb5-87cb-06c1b3ebd077 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:02.264792 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67f9b0-a207-4bb5-87cb-06c1b3ebd077 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:02.266109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67f9b0-a207-4bb5-87cb-06c1b3ebd077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:02.267587 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67f9b0-a207-4bb5-87cb-06c1b3ebd077 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:02.269457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 504/1017] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:02.275890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67f9b0-a207-4bb5-87cb-06c1b3ebd077 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:02.339319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:02.354018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 514/1018] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:02 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:40:02.412699 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67f9b0-a207-4bb5-87cb-06c1b3ebd077 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:02.427824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:02.449525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481042d0-cc23-4e9d-8618-3c386d865aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:02.450155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481042d0-cc23-4e9d-8618-3c386d865aea None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:02.450437 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481042d0-cc23-4e9d-8618-3c386d865aea None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:02.450763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481042d0-cc23-4e9d-8618-3c386d865aea None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:02.451649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481042d0-cc23-4e9d-8618-3c386d865aea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:02.453090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481042d0-cc23-4e9d-8618-3c386d865aea None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:02.480835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 505/1019] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:40:02 2022] POST /identity/v3/projects => generated 311 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:40:02.623624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da75fc-e9f4-48ec-832e-bdf4f8509b24 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:02.738134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da75fc-e9f4-48ec-832e-bdf4f8509b24 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:02.773922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fba39-562b-403b-95c6-869a1651ac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:02.774530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fba39-562b-403b-95c6-869a1651ac69 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:02.774807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fba39-562b-403b-95c6-869a1651ac69 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:02.775077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fba39-562b-403b-95c6-869a1651ac69 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:02.775735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fba39-562b-403b-95c6-869a1651ac69 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:02.777105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fba39-562b-403b-95c6-869a1651ac69 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:02.793900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 515/1020] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:40:02 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:40:04.513593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d05d61-c35a-48a4-86b7-0f53ac0f605f None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:04.514186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d05d61-c35a-48a4-86b7-0f53ac0f605f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:04.514186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d05d61-c35a-48a4-86b7-0f53ac0f605f None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:04.515623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 506/1021] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:04.520633 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6ae88-b4a3-4203-b9d5-801e8b579dc1 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:04.521022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6ae88-b4a3-4203-b9d5-801e8b579dc1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:04.521179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6ae88-b4a3-4203-b9d5-801e8b579dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:04.539537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-45a6ae88-b4a3-4203-b9d5-801e8b579dc1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:04.540728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-45a6ae88-b4a3-4203-b9d5-801e8b579dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:04.593724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:04.623484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 516/1022] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:04.678379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef438753-ba39-4176-9472-423cecadb32e None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:04.678743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef438753-ba39-4176-9472-423cecadb32e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:04.678940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef438753-ba39-4176-9472-423cecadb32e None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:04.680410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 507/1023] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:04.685570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602b65-663c-4438-8cec-c0f23a9d38ab None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:04.685864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602b65-663c-4438-8cec-c0f23a9d38ab None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:04.686141 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602b65-663c-4438-8cec-c0f23a9d38ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:04.704072 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d0602b65-663c-4438-8cec-c0f23a9d38ab None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:04.705250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d0602b65-663c-4438-8cec-c0f23a9d38ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:04.771533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 517/1024] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:04.890022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef438753-ba39-4176-9472-423cecadb32e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:04.969146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936cfaa-0aaf-4067-be3b-d3d40c91314e None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:04.969462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936cfaa-0aaf-4067-be3b-d3d40c91314e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:04.969733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936cfaa-0aaf-4067-be3b-d3d40c91314e None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:04.971203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 518/1025] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:04.975832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c108bd-784d-4243-b3c0-3bd93741f89e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:04.976105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c108bd-784d-4243-b3c0-3bd93741f89e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:04.976373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c108bd-784d-4243-b3c0-3bd93741f89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:04.994045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-06c108bd-784d-4243-b3c0-3bd93741f89e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:04.995213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-06c108bd-784d-4243-b3c0-3bd93741f89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:04.998220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef438753-ba39-4176-9472-423cecadb32e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.037912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b84ab-0924-40e8-811e-be57ddc512c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.038542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b84ab-0924-40e8-811e-be57ddc512c8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.039219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b84ab-0924-40e8-811e-be57ddc512c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.039499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b84ab-0924-40e8-811e-be57ddc512c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.052380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b84ab-0924-40e8-811e-be57ddc512c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-897b84ab-0924-40e8-811e-be57ddc512c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:40:05.057482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:05.058436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:40:05.059280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:05.062758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 508/1026] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 3 14:40:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:05.062758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b84ab-0924-40e8-811e-be57ddc512c8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.066524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 519/1027] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:40:05.067877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c108bd-784d-4243-b3c0-3bd93741f89e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.091678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:05.111091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:05.170799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b84ab-0924-40e8-811e-be57ddc512c8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.174933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c108bd-784d-4243-b3c0-3bd93741f89e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.208277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313f14-6d5a-4114-9962-7a45387c2066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.209258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313f14-6d5a-4114-9962-7a45387c2066 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.209626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313f14-6d5a-4114-9962-7a45387c2066 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.209847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313f14-6d5a-4114-9962-7a45387c2066 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.210951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18313f14-6d5a-4114-9962-7a45387c2066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:05.211212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000526bc-bc98-4618-bc82-4e742e5ce569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.211878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000526bc-bc98-4618-bc82-4e742e5ce569 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.212158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000526bc-bc98-4618-bc82-4e742e5ce569 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.212389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18313f14-6d5a-4114-9962-7a45387c2066 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:05.212487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000526bc-bc98-4618-bc82-4e742e5ce569 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.213069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000526bc-bc98-4618-bc82-4e742e5ce569 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:05.213372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000526bc-bc98-4618-bc82-4e742e5ce569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:05.214145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 509/1028] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:05.214837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000526bc-bc98-4618-bc82-4e742e5ce569 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:05.219451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313f14-6d5a-4114-9962-7a45387c2066 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.229468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 520/1029] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:05.230443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000526bc-bc98-4618-bc82-4e742e5ce569 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.326235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:05.334709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313f14-6d5a-4114-9962-7a45387c2066 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.337748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:05.342724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000526bc-bc98-4618-bc82-4e742e5ce569 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.369887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ce857-35f6-4d3c-920a-e327062e6946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.370534 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ce857-35f6-4d3c-920a-e327062e6946 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.370819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ce857-35f6-4d3c-920a-e327062e6946 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.371094 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ce857-35f6-4d3c-920a-e327062e6946 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.371746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ce857-35f6-4d3c-920a-e327062e6946 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:05.373180 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ce857-35f6-4d3c-920a-e327062e6946 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:05.376437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eaacd6-7cf2-4d75-a5a0-77a545e51b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.377052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaacd6-7cf2-4d75-a5a0-77a545e51b15 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.377324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaacd6-7cf2-4d75-a5a0-77a545e51b15 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.377619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaacd6-7cf2-4d75-a5a0-77a545e51b15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.388123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eaacd6-7cf2-4d75-a5a0-77a545e51b15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:05.389678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eaacd6-7cf2-4d75-a5a0-77a545e51b15 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:05.395852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 510/1030] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:40:05 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:40:05.396679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ce857-35f6-4d3c-920a-e327062e6946 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-c5eaacd6-7cf2-4d75-a5a0-77a545e51b15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:40:05.407317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:05.408357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:40:05.409252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:40:05.409252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:05.409252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:05.417424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 521/1031] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:05.418465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eaacd6-7cf2-4d75-a5a0-77a545e51b15 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.517146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ce857-35f6-4d3c-920a-e327062e6946 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.528307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eaacd6-7cf2-4d75-a5a0-77a545e51b15 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.555545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de05041-dfaa-4f4d-8d6b-8bf197e1c819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.556193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de05041-dfaa-4f4d-8d6b-8bf197e1c819 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.556470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de05041-dfaa-4f4d-8d6b-8bf197e1c819 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.556753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de05041-dfaa-4f4d-8d6b-8bf197e1c819 None admin] PATH_INFO: `/v3/services/image` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.557352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de05041-dfaa-4f4d-8d6b-8bf197e1c819 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:05.558835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de05041-dfaa-4f4d-8d6b-8bf197e1c819 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:05.566102 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e26ee-1cc9-4121-8528-b804b277ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.566813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e26ee-1cc9-4121-8528-b804b277ee20 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.567089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e26ee-1cc9-4121-8528-b804b277ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.567359 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e26ee-1cc9-4121-8528-b804b277ee20 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.568003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e26ee-1cc9-4121-8528-b804b277ee20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:05.568304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e26ee-1cc9-4121-8528-b804b277ee20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-8de05041-dfaa-4f4d-8d6b-8bf197e1c819 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:05.568810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:05.569765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 14:40:05.570670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:05.571568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e26ee-1cc9-4121-8528-b804b277ee20 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:05.571651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 511/1032] 10.222.0.46 () {56 vars in 1105 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:05.575986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de05041-dfaa-4f4d-8d6b-8bf197e1c819 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.585917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 522/1033] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:05.591306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e26ee-1cc9-4121-8528-b804b277ee20 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.594198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:05.609287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:05.700124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de05041-dfaa-4f4d-8d6b-8bf197e1c819 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.702303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e26ee-1cc9-4121-8528-b804b277ee20 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.740733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1d644-b9e7-4e91-83f2-4b43346f8a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.741383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1d644-b9e7-4e91-83f2-4b43346f8a81 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.741691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1d644-b9e7-4e91-83f2-4b43346f8a81 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.741963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1d644-b9e7-4e91-83f2-4b43346f8a81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.746133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e4b36-a882-4771-8819-108bb511060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.746743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e4b36-a882-4771-8819-108bb511060f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.747023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e4b36-a882-4771-8819-108bb511060f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.747354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e4b36-a882-4771-8819-108bb511060f None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.747989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e4b36-a882-4771-8819-108bb511060f None admin] RBAC: Adding query filter params (name=image) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:05.748508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e4b36-a882-4771-8819-108bb511060f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:05.749311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1d644-b9e7-4e91-83f2-4b43346f8a81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:05.750050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e4b36-a882-4771-8819-108bb511060f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:05.750833 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1d644-b9e7-4e91-83f2-4b43346f8a81 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:05.755365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 512/1034] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:05.760997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e4b36-a882-4771-8819-108bb511060f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-b4b1d644-b9e7-4e91-83f2-4b43346f8a81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:40:05.763058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:05.763985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:05.764899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:05.764899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:05.764899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:40:05.764899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:40:05.764899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:40:05.764899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:40:05.764899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:40:05.764899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:40:05.764899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:40:05.764899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:05.770509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 523/1035] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:40:05.772005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1d644-b9e7-4e91-83f2-4b43346f8a81 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.841524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:05.846201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:05.869301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e4b36-a882-4771-8819-108bb511060f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.877130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1d644-b9e7-4e91-83f2-4b43346f8a81 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:05.905898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a15154-2d73-41c8-8df3-6909c1463dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.906597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a15154-2d73-41c8-8df3-6909c1463dc1 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.906879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a15154-2d73-41c8-8df3-6909c1463dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.907157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a15154-2d73-41c8-8df3-6909c1463dc1 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.907782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a15154-2d73-41c8-8df3-6909c1463dc1 None admin] RBAC: Adding query filter params (type=image) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:05.908087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a15154-2d73-41c8-8df3-6909c1463dc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:05.909812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a15154-2d73-41c8-8df3-6909c1463dc1 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:05.911869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520087c-2672-491a-bf03-e6321f457ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:05.912472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520087c-2672-491a-bf03-e6321f457ca7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:05.912747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520087c-2672-491a-bf03-e6321f457ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:05.913018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520087c-2672-491a-bf03-e6321f457ca7 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:05.913767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520087c-2672-491a-bf03-e6321f457ca7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:05.914070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520087c-2672-491a-bf03-e6321f457ca7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:05.919908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 513/1036] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:05.920073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520087c-2672-491a-bf03-e6321f457ca7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:05.920908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a15154-2d73-41c8-8df3-6909c1463dc1 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:05.937801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 524/1037] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:05.943977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520087c-2672-491a-bf03-e6321f457ca7 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:06.027749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a15154-2d73-41c8-8df3-6909c1463dc1 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:06.061736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520087c-2672-491a-bf03-e6321f457ca7 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:06.063536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b170da2-2743-4ba8-95eb-489d6f71a1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:06.064154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b170da2-2743-4ba8-95eb-489d6f71a1d2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:06.064434 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b170da2-2743-4ba8-95eb-489d6f71a1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:06.064703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b170da2-2743-4ba8-95eb-489d6f71a1d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:06.065427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b170da2-2743-4ba8-95eb-489d6f71a1d2 None admin] RBAC: Adding query filter params (interface=public, service_id=0693c83e5ad74ec3a30a47da8a93b856, region_id=RegionOne) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:06.065773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b170da2-2743-4ba8-95eb-489d6f71a1d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:06.067188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b170da2-2743-4ba8-95eb-489d6f71a1d2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:06.072487 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 514/1038] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/endpoints?service_id=0693c83e5ad74ec3a30a47da8a93b856&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:06.078030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b170da2-2743-4ba8-95eb-489d6f71a1d2 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:06.088180 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:06.099880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:06.106581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da41cbd-cc51-45b3-aa1f-0832c157381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:06.107187 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da41cbd-cc51-45b3-aa1f-0832c157381f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:06.107462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da41cbd-cc51-45b3-aa1f-0832c157381f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:06.107729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da41cbd-cc51-45b3-aa1f-0832c157381f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:06.179429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da41cbd-cc51-45b3-aa1f-0832c157381f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:06.180892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da41cbd-cc51-45b3-aa1f-0832c157381f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:06.189044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b170da2-2743-4ba8-95eb-489d6f71a1d2 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:06.225989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a923d-47ab-4218-af7f-42b1a55f20b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:06.226898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a923d-47ab-4218-af7f-42b1a55f20b1 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:06.227362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a923d-47ab-4218-af7f-42b1a55f20b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:06.227841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a923d-47ab-4218-af7f-42b1a55f20b1 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:06.228628 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a923d-47ab-4218-af7f-42b1a55f20b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:06.230244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a923d-47ab-4218-af7f-42b1a55f20b1 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:06.249526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 515/1039] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 3 14:40:06 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:40:06.289652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 525/1040] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 14:40:06.295302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a923d-47ab-4218-af7f-42b1a55f20b1 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:06.380009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:06.416221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a923d-47ab-4218-af7f-42b1a55f20b1 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:06.451077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c29652-e6a5-485b-8ce0-485bff0cec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:06.451712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c29652-e6a5-485b-8ce0-485bff0cec49 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:06.451995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c29652-e6a5-485b-8ce0-485bff0cec49 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:06.452269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c29652-e6a5-485b-8ce0-485bff0cec49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:06.463152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c29652-e6a5-485b-8ce0-485bff0cec49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:06.464644 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c29652-e6a5-485b-8ce0-485bff0cec49 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-04c29652-e6a5-485b-8ce0-485bff0cec49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:40:06.479284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:06.480202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:40:06.481749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:40:06.481749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:06.481749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:06.481749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 516/1041] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:06.486003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da41cbd-cc51-45b3-aa1f-0832c157381f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:06.516137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:06.588443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da41cbd-cc51-45b3-aa1f-0832c157381f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:06.621796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12003d3d-b5f9-4a71-beb0-4c76e3aab29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:06.622399 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12003d3d-b5f9-4a71-beb0-4c76e3aab29f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:06.622675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12003d3d-b5f9-4a71-beb0-4c76e3aab29f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:06.622942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12003d3d-b5f9-4a71-beb0-4c76e3aab29f None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:06.623577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12003d3d-b5f9-4a71-beb0-4c76e3aab29f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:06.623877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12003d3d-b5f9-4a71-beb0-4c76e3aab29f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:06.625242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12003d3d-b5f9-4a71-beb0-4c76e3aab29f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:06.641581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 526/1042] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:06.643163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c29652-e6a5-485b-8ce0-485bff0cec49 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:06.744475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c29652-e6a5-485b-8ce0-485bff0cec49 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:06.766907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:06.780144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c52a3-6c85-43d7-8315-94f9cfe775b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:06.780774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c52a3-6c85-43d7-8315-94f9cfe775b5 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:06.781053 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c52a3-6c85-43d7-8315-94f9cfe775b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:06.781322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c52a3-6c85-43d7-8315-94f9cfe775b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:06.788082 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c52a3-6c85-43d7-8315-94f9cfe775b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:06.789545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c52a3-6c85-43d7-8315-94f9cfe775b5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-485c52a3-6c85-43d7-8315-94f9cfe775b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:40:06.800649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:06.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:06.803047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:40:06.803047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:40:06.803047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:40:06.803047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:40:06.803047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:40:06.803047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:40:06.803047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:40:06.803047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:06.805910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 517/1043] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:40:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:06.808132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12003d3d-b5f9-4a71-beb0-4c76e3aab29f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:06.899253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:06.909773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12003d3d-b5f9-4a71-beb0-4c76e3aab29f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:06.943166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3612673-aa28-456b-80b0-e64f24560320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:06.943773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3612673-aa28-456b-80b0-e64f24560320 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:06.944050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3612673-aa28-456b-80b0-e64f24560320 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:06.944320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3612673-aa28-456b-80b0-e64f24560320 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:06.945006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3612673-aa28-456b-80b0-e64f24560320 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:06.945309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3612673-aa28-456b-80b0-e64f24560320 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:06.946771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3612673-aa28-456b-80b0-e64f24560320 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:06.964024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 527/1044] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:40:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:06.966449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c52a3-6c85-43d7-8315-94f9cfe775b5 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:07.080400 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c52a3-6c85-43d7-8315-94f9cfe775b5 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:07.113896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405dffcd-1148-49ad-abe9-57accaaba53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:07.114507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dffcd-1148-49ad-abe9-57accaaba53a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:07.114782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dffcd-1148-49ad-abe9-57accaaba53a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:07.115048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dffcd-1148-49ad-abe9-57accaaba53a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:07.115768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405dffcd-1148-49ad-abe9-57accaaba53a None admin] RBAC: Adding query filter params (role.id=662ff20775c14ad69b5a84324701fcf1, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=1a4a95ae5ee84f259ffcd7528b0fa505) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:07.116072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405dffcd-1148-49ad-abe9-57accaaba53a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:07.117478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405dffcd-1148-49ad-abe9-57accaaba53a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:07.129873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 518/1045] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:40:06 2022] GET /identity/v3/role_assignments?user.id=1a4a95ae5ee84f259ffcd7528b0fa505&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=662ff20775c14ad69b5a84324701fcf1 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:07.651217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd24d3-9a95-481e-bfb9-217ce240d7e6 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:07.651726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd24d3-9a95-481e-bfb9-217ce240d7e6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:07.651726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd24d3-9a95-481e-bfb9-217ce240d7e6 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:07.653180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 528/1046] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:07.658254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28609539-b4b1-4771-b767-3293cf1c0ada None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:07.658668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28609539-b4b1-4771-b767-3293cf1c0ada None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:07.658833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28609539-b4b1-4771-b767-3293cf1c0ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:07.678149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-28609539-b4b1-4771-b767-3293cf1c0ada None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:07.679338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-28609539-b4b1-4771-b767-3293cf1c0ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:07.722620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:07.753541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 519/1047] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:08.024459 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd24d3-9a95-481e-bfb9-217ce240d7e6 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:08.127008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd24d3-9a95-481e-bfb9-217ce240d7e6 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:08.161077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85d11a-56ad-4287-9873-755656ce9b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:08.161723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85d11a-56ad-4287-9873-755656ce9b21 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:08.162005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85d11a-56ad-4287-9873-755656ce9b21 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:08.162273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85d11a-56ad-4287-9873-755656ce9b21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:08.172250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85d11a-56ad-4287-9873-755656ce9b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-1d85d11a-56ad-4287-9873-755656ce9b21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:40:08.177139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:08.178225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:08.179037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:08.180310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 529/1048] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:40:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:08.184181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28609539-b4b1-4771-b767-3293cf1c0ada None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:08.286364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28609539-b4b1-4771-b767-3293cf1c0ada None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:08.325450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381fe05f-6a29-407d-b245-222977f316d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:08.326104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fe05f-6a29-407d-b245-222977f316d5 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:08.326383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fe05f-6a29-407d-b245-222977f316d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:08.326655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fe05f-6a29-407d-b245-222977f316d5 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:08.327351 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381fe05f-6a29-407d-b245-222977f316d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:08.327737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381fe05f-6a29-407d-b245-222977f316d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:08.329148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381fe05f-6a29-407d-b245-222977f316d5 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:08.343688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 520/1049] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:40:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:08.344899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85d11a-56ad-4287-9873-755656ce9b21 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:08.359218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:08.423253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0cf0d-60b4-4247-8b59-ac8c8ba91d0a None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:08.423563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0cf0d-60b4-4247-8b59-ac8c8ba91d0a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:08.423809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0cf0d-60b4-4247-8b59-ac8c8ba91d0a None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:08.425268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 521/1050] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:40:08.429961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aa6e3-d76a-4acc-a5cf-39ebe776a5e4 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:08.430246 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aa6e3-d76a-4acc-a5cf-39ebe776a5e4 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:08.430521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aa6e3-d76a-4acc-a5cf-39ebe776a5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:08.448479 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-fa3aa6e3-d76a-4acc-a5cf-39ebe776a5e4 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:08.449668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85d11a-56ad-4287-9873-755656ce9b21 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:08.449794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-fa3aa6e3-d76a-4acc-a5cf-39ebe776a5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:08.484063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:08.485445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044eb57-da25-4e8d-aad6-1701a741e40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:08.485793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044eb57-da25-4e8d-aad6-1701a741e40a None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:08.486059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044eb57-da25-4e8d-aad6-1701a741e40a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:08.486303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044eb57-da25-4e8d-aad6-1701a741e40a None admin] PATH_INFO: `/v3/users/0f420bf7c9bb46ff8812779f03b5a892` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:08.499861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044eb57-da25-4e8d-aad6-1701a741e40a None admin] RBAC: Authorizing `identity:get_user(user_id=0f420bf7c9bb46ff8812779f03b5a892)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:08.501296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044eb57-da25-4e8d-aad6-1701a741e40a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:08.507484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 530/1051] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:08 2022] GET /identity/v3/users/0f420bf7c9bb46ff8812779f03b5a892 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:08.508425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044eb57-da25-4e8d-aad6-1701a741e40a None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:08.530747 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 522/1052] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:40:08.578646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:08.614374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044eb57-da25-4e8d-aad6-1701a741e40a None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:08.648971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba93b89-38ef-43f4-b1af-33b5bad50bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:08.649611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba93b89-38ef-43f4-b1af-33b5bad50bc6 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:08.649892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba93b89-38ef-43f4-b1af-33b5bad50bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:08.650163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba93b89-38ef-43f4-b1af-33b5bad50bc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:08.718558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba93b89-38ef-43f4-b1af-33b5bad50bc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:08.720005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba93b89-38ef-43f4-b1af-33b5bad50bc6 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:08.783537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:08.794288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aa6e3-d76a-4acc-a5cf-39ebe776a5e4 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:08.822072 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 531/1053] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:08 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 14:40:08.828271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba93b89-38ef-43f4-b1af-33b5bad50bc6 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:08.902212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aa6e3-d76a-4acc-a5cf-39ebe776a5e4 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:08.928977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba93b89-38ef-43f4-b1af-33b5bad50bc6 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:08.937651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89fc4c-9ae4-4bd8-98c6-51e9ca7092ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:08.938626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89fc4c-9ae4-4bd8-98c6-51e9ca7092ff None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:08.939109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89fc4c-9ae4-4bd8-98c6-51e9ca7092ff None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:08.939578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89fc4c-9ae4-4bd8-98c6-51e9ca7092ff None admin] PATH_INFO: `/v3/services/image` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:08.940356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89fc4c-9ae4-4bd8-98c6-51e9ca7092ff None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:08.942452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89fc4c-9ae4-4bd8-98c6-51e9ca7092ff None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-5f89fc4c-9ae4-4bd8-98c6-51e9ca7092ff None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:08.951348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:08.952332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 14:40:08.953272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:08.954696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 523/1054] 10.222.0.46 () {56 vars in 1105 bytes} [Mon Oct 3 14:40:08 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:40:08.960937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89fc4c-9ae4-4bd8-98c6-51e9ca7092ff None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:08.965199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5738cd0-7364-467b-a6d3-423fbf2c232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:08.965860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5738cd0-7364-467b-a6d3-423fbf2c232c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:08.966148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5738cd0-7364-467b-a6d3-423fbf2c232c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:08.966423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5738cd0-7364-467b-a6d3-423fbf2c232c None admin] PATH_INFO: `/v3/projects/50363ca61fb446ccb560ed0bad95984d` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:08.967675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5738cd0-7364-467b-a6d3-423fbf2c232c None admin] RBAC: Authorizing `identity:get_project(project_id=50363ca61fb446ccb560ed0bad95984d)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:08.969151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5738cd0-7364-467b-a6d3-423fbf2c232c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:08.970848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 532/1055] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:40:08 2022] GET /identity/v3/projects/50363ca61fb446ccb560ed0bad95984d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:08.978239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5738cd0-7364-467b-a6d3-423fbf2c232c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:09.000360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:09.023020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:09.074111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89fc4c-9ae4-4bd8-98c6-51e9ca7092ff None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:09.090395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5738cd0-7364-467b-a6d3-423fbf2c232c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:09.109577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da3953-a7fc-4259-adb9-ce0f3cb59276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:09.110187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da3953-a7fc-4259-adb9-ce0f3cb59276 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:09.110469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da3953-a7fc-4259-adb9-ce0f3cb59276 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:09.110763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da3953-a7fc-4259-adb9-ce0f3cb59276 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:09.111404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da3953-a7fc-4259-adb9-ce0f3cb59276 None admin] RBAC: Adding query filter params (name=image) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:09.111710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da3953-a7fc-4259-adb9-ce0f3cb59276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:09.113390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da3953-a7fc-4259-adb9-ce0f3cb59276 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:09.120862 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 524/1056] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:09.126907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da3953-a7fc-4259-adb9-ce0f3cb59276 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:09.128276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe7147-ecfb-4545-b35e-52ad24ed5193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:09.129162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe7147-ecfb-4545-b35e-52ad24ed5193 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:09.129654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe7147-ecfb-4545-b35e-52ad24ed5193 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:09.130094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe7147-ecfb-4545-b35e-52ad24ed5193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:09.130998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe7147-ecfb-4545-b35e-52ad24ed5193 None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=50363ca61fb446ccb560ed0bad95984d, user.id=0f420bf7c9bb46ff8812779f03b5a892) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:09.131305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe7147-ecfb-4545-b35e-52ad24ed5193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:09.132781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe7147-ecfb-4545-b35e-52ad24ed5193 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:09.149548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 533/1057] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:40:08 2022] GET /identity/v3/role_assignments?user.id=0f420bf7c9bb46ff8812779f03b5a892&scope.project.id=50363ca61fb446ccb560ed0bad95984d&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:09.243040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:09.246126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173f8ff-b5c3-40ea-9859-e85e57625994 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:09.246400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173f8ff-b5c3-40ea-9859-e85e57625994 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:09.246673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173f8ff-b5c3-40ea-9859-e85e57625994 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:09.248033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 534/1058] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:09.251396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da3953-a7fc-4259-adb9-ce0f3cb59276 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:09.253439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddd5f1-7488-4bba-9656-0ac90cc0a58a None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:09.253753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddd5f1-7488-4bba-9656-0ac90cc0a58a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:09.254071 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddd5f1-7488-4bba-9656-0ac90cc0a58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:09.273181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-29ddd5f1-7488-4bba-9656-0ac90cc0a58a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:09.274421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-29ddd5f1-7488-4bba-9656-0ac90cc0a58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:09.291795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb110f4-b76d-4da2-8502-178b38795226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:09.292418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb110f4-b76d-4da2-8502-178b38795226 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:09.292698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb110f4-b76d-4da2-8502-178b38795226 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:09.292976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb110f4-b76d-4da2-8502-178b38795226 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:09.293663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb110f4-b76d-4da2-8502-178b38795226 None admin] RBAC: Adding query filter params (type=image) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:09.293969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb110f4-b76d-4da2-8502-178b38795226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:09.295372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb110f4-b76d-4da2-8502-178b38795226 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:09.305538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 525/1059] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:09.308473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb110f4-b76d-4da2-8502-178b38795226 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:09.365531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 535/1060] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:40:09.417716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb110f4-b76d-4da2-8502-178b38795226 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:09.451218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112aebc5-ab59-41bf-87d8-7161cd0fff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:09.451820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aebc5-ab59-41bf-87d8-7161cd0fff7d None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:09.452099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aebc5-ab59-41bf-87d8-7161cd0fff7d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:09.452368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aebc5-ab59-41bf-87d8-7161cd0fff7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:09.453030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112aebc5-ab59-41bf-87d8-7161cd0fff7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:09.454428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112aebc5-ab59-41bf-87d8-7161cd0fff7d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:09.484814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 526/1061] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:40:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:40:09.630693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddd5f1-7488-4bba-9656-0ac90cc0a58a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:09.661957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:09.734098 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddd5f1-7488-4bba-9656-0ac90cc0a58a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:09.775054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602fd31-5557-43ff-9394-2da75296495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:09.775688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602fd31-5557-43ff-9394-2da75296495d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:09.775965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602fd31-5557-43ff-9394-2da75296495d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:09.776231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602fd31-5557-43ff-9394-2da75296495d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:09.786396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602fd31-5557-43ff-9394-2da75296495d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-5602fd31-5557-43ff-9394-2da75296495d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:40:09.791190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:09.792094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:09.793441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 536/1062] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 3 14:40:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:09.798719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112aebc5-ab59-41bf-87d8-7161cd0fff7d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:09.818110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:09.902588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112aebc5-ab59-41bf-87d8-7161cd0fff7d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:09.936823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb5d9e-4a3a-4102-935b-45b656b25a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:09.937430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb5d9e-4a3a-4102-935b-45b656b25a0e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:09.937735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb5d9e-4a3a-4102-935b-45b656b25a0e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:09.938008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb5d9e-4a3a-4102-935b-45b656b25a0e None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:09.938671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb5d9e-4a3a-4102-935b-45b656b25a0e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:09.938972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb5d9e-4a3a-4102-935b-45b656b25a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:09.940404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb5d9e-4a3a-4102-935b-45b656b25a0e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:09.953021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 527/1063] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 3 14:40:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:09.954359 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602fd31-5557-43ff-9394-2da75296495d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:10.045638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:10.061135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602fd31-5557-43ff-9394-2da75296495d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:10.095031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13960be-09b0-493a-b579-596742b47b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:10.095649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13960be-09b0-493a-b579-596742b47b5c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:10.095927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13960be-09b0-493a-b579-596742b47b5c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:10.096249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13960be-09b0-493a-b579-596742b47b5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:10.106995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13960be-09b0-493a-b579-596742b47b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:10.108443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13960be-09b0-493a-b579-596742b47b5c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-a13960be-09b0-493a-b579-596742b47b5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:40:10.123214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:10.124231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:40:10.129930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:40:10.129930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:10.129930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:10.129930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 537/1064] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:10.131765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb5d9e-4a3a-4102-935b-45b656b25a0e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:10.210433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:10.233468 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb5d9e-4a3a-4102-935b-45b656b25a0e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:10.267929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd847d-a048-45b7-80af-daafd814882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:10.268537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd847d-a048-45b7-80af-daafd814882b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:10.268818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd847d-a048-45b7-80af-daafd814882b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:10.269089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd847d-a048-45b7-80af-daafd814882b None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:10.270450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd847d-a048-45b7-80af-daafd814882b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:10.270757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd847d-a048-45b7-80af-daafd814882b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:10.272126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd847d-a048-45b7-80af-daafd814882b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:10.284269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 528/1065] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:10.292216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13960be-09b0-493a-b579-596742b47b5c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:10.401127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13960be-09b0-493a-b579-596742b47b5c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:10.436153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71523e2b-0630-439f-a32a-2543346b3787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:10.436780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71523e2b-0630-439f-a32a-2543346b3787 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:10.437059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71523e2b-0630-439f-a32a-2543346b3787 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:10.437328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71523e2b-0630-439f-a32a-2543346b3787 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:10.445484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71523e2b-0630-439f-a32a-2543346b3787 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:10.446950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71523e2b-0630-439f-a32a-2543346b3787 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:10.448625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-71523e2b-0630-439f-a32a-2543346b3787 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:40:10.458593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:10.459609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:10.460598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 538/1066] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 3 14:40:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:10.465645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd847d-a048-45b7-80af-daafd814882b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:10.566288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd847d-a048-45b7-80af-daafd814882b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:10.600122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006a5e2-98aa-4cc1-8046-52349e478553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:10.600726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006a5e2-98aa-4cc1-8046-52349e478553 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:10.601006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006a5e2-98aa-4cc1-8046-52349e478553 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:10.601279 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006a5e2-98aa-4cc1-8046-52349e478553 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:10.602039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006a5e2-98aa-4cc1-8046-52349e478553 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:10.602348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006a5e2-98aa-4cc1-8046-52349e478553 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:10.603772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006a5e2-98aa-4cc1-8046-52349e478553 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:10.611588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:10.622958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 529/1067] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 3 14:40:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:10.629578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71523e2b-0630-439f-a32a-2543346b3787 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:10.732031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71523e2b-0630-439f-a32a-2543346b3787 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:10.766815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448979d-7b68-432b-993c-9061f8823d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:10.767437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448979d-7b68-432b-993c-9061f8823d9e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:10.767711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448979d-7b68-432b-993c-9061f8823d9e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:10.767978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448979d-7b68-432b-993c-9061f8823d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:10.779325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448979d-7b68-432b-993c-9061f8823d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:10.780831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448979d-7b68-432b-993c-9061f8823d9e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-1448979d-7b68-432b-993c-9061f8823d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:40:10.795225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:10.796160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:40:10.797056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:40:10.797056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:10.797056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:10.799435 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 539/1068] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:10.803199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006a5e2-98aa-4cc1-8046-52349e478553 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:10.903516 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006a5e2-98aa-4cc1-8046-52349e478553 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:10.938148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a18d91-50a9-4ee2-abee-69c356f400a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:10.938745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a18d91-50a9-4ee2-abee-69c356f400a4 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:10.939020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a18d91-50a9-4ee2-abee-69c356f400a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:10.939288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a18d91-50a9-4ee2-abee-69c356f400a4 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:10.939925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a18d91-50a9-4ee2-abee-69c356f400a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:10.940226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a18d91-50a9-4ee2-abee-69c356f400a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:10.941629 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a18d91-50a9-4ee2-abee-69c356f400a4 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:10.953022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 530/1069] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:10.958380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448979d-7b68-432b-993c-9061f8823d9e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:11.010238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:11.061686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448979d-7b68-432b-993c-9061f8823d9e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:11.094207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b192c37-43f0-4439-b12f-e3f2aa1bbc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:11.094829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b192c37-43f0-4439-b12f-e3f2aa1bbc9d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:11.095101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b192c37-43f0-4439-b12f-e3f2aa1bbc9d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:11.095363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b192c37-43f0-4439-b12f-e3f2aa1bbc9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:11.102492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b192c37-43f0-4439-b12f-e3f2aa1bbc9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:11.103921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b192c37-43f0-4439-b12f-e3f2aa1bbc9d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-4b192c37-43f0-4439-b12f-e3f2aa1bbc9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:40:11.115695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:11.116599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:11.117479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:40:11.117479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:40:11.117479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:40:11.117479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:40:11.117479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:40:11.117479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:40:11.117479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:40:11.117479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:11.123014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 540/1070] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:40:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:11.123137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a18d91-50a9-4ee2-abee-69c356f400a4 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:11.182908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:11.228818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a18d91-50a9-4ee2-abee-69c356f400a4 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:11.262825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f051e-abb7-41af-86fb-e3c5d04df49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:11.263356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f051e-abb7-41af-86fb-e3c5d04df49f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:11.263630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f051e-abb7-41af-86fb-e3c5d04df49f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:11.263898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f051e-abb7-41af-86fb-e3c5d04df49f None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:11.264581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f051e-abb7-41af-86fb-e3c5d04df49f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:11.264881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f051e-abb7-41af-86fb-e3c5d04df49f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:11.266314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f051e-abb7-41af-86fb-e3c5d04df49f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:11.281052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 531/1071] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:40:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:11.283391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daa8c6-d662-4712-83e3-96493e8622d4 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:11.283675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daa8c6-d662-4712-83e3-96493e8622d4 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:11.283949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daa8c6-d662-4712-83e3-96493e8622d4 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:11.285982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 541/1072] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:11.287508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f051e-abb7-41af-86fb-e3c5d04df49f None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:11.292798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525296c5-5ead-4e32-84f8-22fb566de06a None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:11.292798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525296c5-5ead-4e32-84f8-22fb566de06a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:11.292798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525296c5-5ead-4e32-84f8-22fb566de06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:11.309774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-525296c5-5ead-4e32-84f8-22fb566de06a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:11.310941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-525296c5-5ead-4e32-84f8-22fb566de06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:11.382807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 542/1073] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:40:11.394949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f051e-abb7-41af-86fb-e3c5d04df49f None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:11.411700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:11.430230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397354bb-99b6-4cf7-b0a0-87e5e79e2568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:11.430884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397354bb-99b6-4cf7-b0a0-87e5e79e2568 None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:11.431159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397354bb-99b6-4cf7-b0a0-87e5e79e2568 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:11.431428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397354bb-99b6-4cf7-b0a0-87e5e79e2568 None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/1a4a95ae5ee84f259ffcd7528b0fa505/roles/662ff20775c14ad69b5a84324701fcf1` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:11.433547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397354bb-99b6-4cf7-b0a0-87e5e79e2568 None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=1a4a95ae5ee84f259ffcd7528b0fa505, role_id=662ff20775c14ad69b5a84324701fcf1)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:11.434969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397354bb-99b6-4cf7-b0a0-87e5e79e2568 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:11.447406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 532/1074] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:40:11 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/1a4a95ae5ee84f259ffcd7528b0fa505/roles/662ff20775c14ad69b5a84324701fcf1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:40:11.647504 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525296c5-5ead-4e32-84f8-22fb566de06a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:11.656995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec6785-99e0-4924-b603-bab827aae4f6 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:11.657281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec6785-99e0-4924-b603-bab827aae4f6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:11.657592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec6785-99e0-4924-b603-bab827aae4f6 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:11.658923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 533/1075] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:11.664627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52be0cb-db72-4daa-a1e4-858c07fd3187 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:11.664910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52be0cb-db72-4daa-a1e4-858c07fd3187 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:11.665187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52be0cb-db72-4daa-a1e4-858c07fd3187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:11.683596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f52be0cb-db72-4daa-a1e4-858c07fd3187 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:11.684776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f52be0cb-db72-4daa-a1e4-858c07fd3187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:11.688648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:11.748865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525296c5-5ead-4e32-84f8-22fb566de06a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:11.760070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 534/1076] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:11.784188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612960b-ceec-48b5-9cfe-f2c800a1faa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:11.784846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612960b-ceec-48b5-9cfe-f2c800a1faa1 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:11.785642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612960b-ceec-48b5-9cfe-f2c800a1faa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:11.785911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612960b-ceec-48b5-9cfe-f2c800a1faa1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:11.795856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612960b-ceec-48b5-9cfe-f2c800a1faa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-8612960b-ceec-48b5-9cfe-f2c800a1faa1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:40:11.800563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:11.802165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:11.803008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:11.804476 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 543/1077] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:40:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:40:11.808054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52be0cb-db72-4daa-a1e4-858c07fd3187 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:11.907691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52be0cb-db72-4daa-a1e4-858c07fd3187 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:11.914972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:11.944311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915d7de-7587-4dee-b0e9-fd98d9689d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:11.945198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915d7de-7587-4dee-b0e9-fd98d9689d3e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:11.945485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915d7de-7587-4dee-b0e9-fd98d9689d3e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:11.945776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915d7de-7587-4dee-b0e9-fd98d9689d3e None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:11.946435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915d7de-7587-4dee-b0e9-fd98d9689d3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:11.946734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915d7de-7587-4dee-b0e9-fd98d9689d3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:11.948081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915d7de-7587-4dee-b0e9-fd98d9689d3e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:11.961517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 535/1078] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:40:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:11.962851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612960b-ceec-48b5-9cfe-f2c800a1faa1 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:12.028088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915d7de-7587-4dee-b0e9-fd98d9689d3e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:12.060559 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:12.063813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612960b-ceec-48b5-9cfe-f2c800a1faa1 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:12.099215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87555c82-d6fc-40b1-bc55-d8dd24163be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:12.099848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87555c82-d6fc-40b1-bc55-d8dd24163be3 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:12.100149 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87555c82-d6fc-40b1-bc55-d8dd24163be3 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:12.100382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87555c82-d6fc-40b1-bc55-d8dd24163be3 None admin] PATH_INFO: `/v3/users/0f420bf7c9bb46ff8812779f03b5a892` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:12.101514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87555c82-d6fc-40b1-bc55-d8dd24163be3 None admin] RBAC: Authorizing `identity:get_user(user_id=0f420bf7c9bb46ff8812779f03b5a892)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:12.102948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87555c82-d6fc-40b1-bc55-d8dd24163be3 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:12.104678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 544/1079] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:11 2022] GET /identity/v3/users/0f420bf7c9bb46ff8812779f03b5a892 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:12.110064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87555c82-d6fc-40b1-bc55-d8dd24163be3 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:12.133768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915d7de-7587-4dee-b0e9-fd98d9689d3e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:12.170600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73b525-b71e-411c-8379-e28d306574d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:12.171130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b525-b71e-411c-8379-e28d306574d1 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:12.171450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b525-b71e-411c-8379-e28d306574d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:12.171676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73b525-b71e-411c-8379-e28d306574d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:12.209568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:12.211383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87555c82-d6fc-40b1-bc55-d8dd24163be3 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:12.243901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73b525-b71e-411c-8379-e28d306574d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:12.245359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73b525-b71e-411c-8379-e28d306574d1 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:12.246260 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb55c22-7fde-4398-9078-167bc2ff6479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:12.246886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb55c22-7fde-4398-9078-167bc2ff6479 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:12.247177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb55c22-7fde-4398-9078-167bc2ff6479 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:12.247430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb55c22-7fde-4398-9078-167bc2ff6479 None admin] PATH_INFO: `/v3/projects/50363ca61fb446ccb560ed0bad95984d` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:12.248585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb55c22-7fde-4398-9078-167bc2ff6479 None admin] RBAC: Authorizing `identity:get_project(project_id=50363ca61fb446ccb560ed0bad95984d)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:12.249976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb55c22-7fde-4398-9078-167bc2ff6479 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:12.251760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 545/1080] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:40:12 2022] GET /identity/v3/projects/50363ca61fb446ccb560ed0bad95984d => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:12.257350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb55c22-7fde-4398-9078-167bc2ff6479 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:12.298942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:12.364884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 536/1081] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 14:40:12.370057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb55c22-7fde-4398-9078-167bc2ff6479 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:12.370400 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73b525-b71e-411c-8379-e28d306574d1 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:12.406099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7991c-0b37-461c-bd95-b9f681453dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:12.406747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7991c-0b37-461c-bd95-b9f681453dba None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:12.407021 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7991c-0b37-461c-bd95-b9f681453dba None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:12.407287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7991c-0b37-461c-bd95-b9f681453dba None admin] PATH_INFO: `/v3/projects/50363ca61fb446ccb560ed0bad95984d/users/0f420bf7c9bb46ff8812779f03b5a892/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:12.409145 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7991c-0b37-461c-bd95-b9f681453dba None admin] RBAC: Authorizing `identity:create_grant(project_id=50363ca61fb446ccb560ed0bad95984d, user_id=0f420bf7c9bb46ff8812779f03b5a892, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:12.411132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7991c-0b37-461c-bd95-b9f681453dba None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:12.424624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 546/1082] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:40:12 2022] PUT /identity/v3/projects/50363ca61fb446ccb560ed0bad95984d/users/0f420bf7c9bb46ff8812779f03b5a892/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:40:12.440624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:12.478554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73b525-b71e-411c-8379-e28d306574d1 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:12.517100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4ce5f-138e-4de2-a27d-0941b6541c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:12.517778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ce5f-138e-4de2-a27d-0941b6541c6f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:12.518057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ce5f-138e-4de2-a27d-0941b6541c6f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:12.518325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ce5f-138e-4de2-a27d-0941b6541c6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:12.529545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4ce5f-138e-4de2-a27d-0941b6541c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:12.531014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4ce5f-138e-4de2-a27d-0941b6541c6f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-57b4ce5f-138e-4de2-a27d-0941b6541c6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:40:12.546994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:12.548044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:40:12.548941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:40:12.548941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:12.548941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:12.548941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 537/1083] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:40:12.554782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7991c-0b37-461c-bd95-b9f681453dba None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:12.655481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:12.664169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7991c-0b37-461c-bd95-b9f681453dba None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:12.698780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8afa1a-c7f5-457f-8918-2f591e3afede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:12.699300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8afa1a-c7f5-457f-8918-2f591e3afede None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:12.699572 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8afa1a-c7f5-457f-8918-2f591e3afede None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:12.699836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8afa1a-c7f5-457f-8918-2f591e3afede None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:12.700476 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8afa1a-c7f5-457f-8918-2f591e3afede None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:12.700772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8afa1a-c7f5-457f-8918-2f591e3afede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:12.702152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8afa1a-c7f5-457f-8918-2f591e3afede None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:12.714590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 547/1084] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:13.626207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5975-ab62-4520-96de-aa968bb4062d None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:13.626696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5975-ab62-4520-96de-aa968bb4062d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:13.626696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5975-ab62-4520-96de-aa968bb4062d None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:13.627996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 538/1085] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:13.633334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f2f6d-01d5-4e2c-a3ac-a6f7099d1654 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:13.633740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f2f6d-01d5-4e2c-a3ac-a6f7099d1654 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:13.633925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f2f6d-01d5-4e2c-a3ac-a6f7099d1654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:13.652153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-7b6f2f6d-01d5-4e2c-a3ac-a6f7099d1654 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:13.653344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7b6f2f6d-01d5-4e2c-a3ac-a6f7099d1654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:13.717418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 548/1086] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:13.986406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a5975-ab62-4520-96de-aa968bb4062d None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:14.085096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a5975-ab62-4520-96de-aa968bb4062d None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:14.104290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:14.118614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e7f41-abd2-4f1a-be83-4842077ec3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:14.119239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e7f41-abd2-4f1a-be83-4842077ec3ee None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:14.119520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e7f41-abd2-4f1a-be83-4842077ec3ee None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:14.119794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e7f41-abd2-4f1a-be83-4842077ec3ee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:14.130542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e7f41-abd2-4f1a-be83-4842077ec3ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-919e7f41-abd2-4f1a-be83-4842077ec3ee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:40:14.135284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:14.136350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:14.137202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 539/1087] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 3 14:40:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:14.142355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f2f6d-01d5-4e2c-a3ac-a6f7099d1654 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:14.224765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:14.240228 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f2f6d-01d5-4e2c-a3ac-a6f7099d1654 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:14.274986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf282a5-cc6a-40ca-9c23-e58fc5800eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:14.275578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf282a5-cc6a-40ca-9c23-e58fc5800eab None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:14.278652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf282a5-cc6a-40ca-9c23-e58fc5800eab None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:14.278652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf282a5-cc6a-40ca-9c23-e58fc5800eab None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:14.278652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf282a5-cc6a-40ca-9c23-e58fc5800eab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:14.278652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf282a5-cc6a-40ca-9c23-e58fc5800eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:14.278652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf282a5-cc6a-40ca-9c23-e58fc5800eab None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:14.286155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 549/1088] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 3 14:40:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:14.292036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e7f41-abd2-4f1a-be83-4842077ec3ee None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:14.389650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e7f41-abd2-4f1a-be83-4842077ec3ee None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:14.421310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9746d5-8820-493e-b133-7101d51985d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:14.421954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9746d5-8820-493e-b133-7101d51985d2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:14.422234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9746d5-8820-493e-b133-7101d51985d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:14.422504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9746d5-8820-493e-b133-7101d51985d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:14.433028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9746d5-8820-493e-b133-7101d51985d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:14.434495 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9746d5-8820-493e-b133-7101d51985d2 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-1d9746d5-8820-493e-b133-7101d51985d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:40:14.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:14.449775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:40:14.450729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:40:14.450729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:14.450729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:14.450729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 540/1089] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:40:14.455569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf282a5-cc6a-40ca-9c23-e58fc5800eab None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:14.555139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf282a5-cc6a-40ca-9c23-e58fc5800eab None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:14.579244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c78db-eb82-4e63-bd62-3e39d9869ad6 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:14.579532 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c78db-eb82-4e63-bd62-3e39d9869ad6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:14.579801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c78db-eb82-4e63-bd62-3e39d9869ad6 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:14.581122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 541/1090] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:40:14.586911 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda9fb1-bf47-4329-800a-5f6d0beaf294 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:14.587198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda9fb1-bf47-4329-800a-5f6d0beaf294 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:14.587477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda9fb1-bf47-4329-800a-5f6d0beaf294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:14.588797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8a23e-062b-454d-a9b6-01c6a49bfa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:14.589406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8a23e-062b-454d-a9b6-01c6a49bfa42 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:14.589714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8a23e-062b-454d-a9b6-01c6a49bfa42 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:14.589990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8a23e-062b-454d-a9b6-01c6a49bfa42 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:14.590634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8a23e-062b-454d-a9b6-01c6a49bfa42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:14.590934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8a23e-062b-454d-a9b6-01c6a49bfa42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:14.592289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8a23e-062b-454d-a9b6-01c6a49bfa42 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:14.604608 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 550/1091] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:14.607159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9bda9fb1-bf47-4329-800a-5f6d0beaf294 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:14.608375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9bda9fb1-bf47-4329-800a-5f6d0beaf294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:14.610184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8a23e-062b-454d-a9b6-01c6a49bfa42 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:14.620665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:14.634164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:14.675346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 542/1092] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:40:14.709120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8a23e-062b-454d-a9b6-01c6a49bfa42 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:14.741839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f23a4e-0d78-48d3-bde6-63645b54abd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:14.742457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f23a4e-0d78-48d3-bde6-63645b54abd9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:14.742736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f23a4e-0d78-48d3-bde6-63645b54abd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:14.743009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f23a4e-0d78-48d3-bde6-63645b54abd9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:14.749882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f23a4e-0d78-48d3-bde6-63645b54abd9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:14.751363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f23a4e-0d78-48d3-bde6-63645b54abd9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-e1f23a4e-0d78-48d3-bde6-63645b54abd9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:40:14.763275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:14.764387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:14.765334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 551/1093] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 3 14:40:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:40:14.770831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda9fb1-bf47-4329-800a-5f6d0beaf294 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:14.865418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda9fb1-bf47-4329-800a-5f6d0beaf294 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:14.897809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801f395-e0d9-40d3-a0db-5c6c87876c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:14.898410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801f395-e0d9-40d3-a0db-5c6c87876c14 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:14.898688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801f395-e0d9-40d3-a0db-5c6c87876c14 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:14.898959 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801f395-e0d9-40d3-a0db-5c6c87876c14 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:14.899681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801f395-e0d9-40d3-a0db-5c6c87876c14 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:14.899985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801f395-e0d9-40d3-a0db-5c6c87876c14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:14.901401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801f395-e0d9-40d3-a0db-5c6c87876c14 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:14.918394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 543/1094] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 3 14:40:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:14.924059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f23a4e-0d78-48d3-bde6-63645b54abd9 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:14.939979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801f395-e0d9-40d3-a0db-5c6c87876c14 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:14.960557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:14.990283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:15.023350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f23a4e-0d78-48d3-bde6-63645b54abd9 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.038300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801f395-e0d9-40d3-a0db-5c6c87876c14 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.056586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97416a82-d149-4fc0-8e5d-3b77208c794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.057187 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97416a82-d149-4fc0-8e5d-3b77208c794d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.057464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97416a82-d149-4fc0-8e5d-3b77208c794d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.057762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97416a82-d149-4fc0-8e5d-3b77208c794d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.071730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0c3a4-096f-4836-b49f-0bd283497d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.072383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c3a4-096f-4836-b49f-0bd283497d1b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.072703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c3a4-096f-4836-b49f-0bd283497d1b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.073002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c3a4-096f-4836-b49f-0bd283497d1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.084110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0c3a4-096f-4836-b49f-0bd283497d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-2be0c3a4-096f-4836-b49f-0bd283497d1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:40:15.088809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:15.090003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:15.090893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 544/1095] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:40:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:40:15.095781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0c3a4-096f-4836-b49f-0bd283497d1b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:15.123788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97416a82-d149-4fc0-8e5d-3b77208c794d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:15.125158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97416a82-d149-4fc0-8e5d-3b77208c794d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:15.175009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:15.192021 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:15.193371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0c3a4-096f-4836-b49f-0bd283497d1b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.227251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 552/1096] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:40:15.232710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02def5-8112-4bda-8d41-86d9d9fc6858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.233037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97416a82-d149-4fc0-8e5d-3b77208c794d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:15.233316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02def5-8112-4bda-8d41-86d9d9fc6858 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.233620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02def5-8112-4bda-8d41-86d9d9fc6858 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.233893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02def5-8112-4bda-8d41-86d9d9fc6858 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.234557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02def5-8112-4bda-8d41-86d9d9fc6858 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:15.234860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02def5-8112-4bda-8d41-86d9d9fc6858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:15.236215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02def5-8112-4bda-8d41-86d9d9fc6858 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:15.244520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 545/1097] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:40:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:15.250253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02def5-8112-4bda-8d41-86d9d9fc6858 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:15.334587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97416a82-d149-4fc0-8e5d-3b77208c794d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.347678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02def5-8112-4bda-8d41-86d9d9fc6858 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.367668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b222cda-a707-423d-a020-ae499e36ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.368290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b222cda-a707-423d-a020-ae499e36ac16 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.368569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b222cda-a707-423d-a020-ae499e36ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.368841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b222cda-a707-423d-a020-ae499e36ac16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.379290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b222cda-a707-423d-a020-ae499e36ac16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:15.380660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b222cda-a707-423d-a020-ae499e36ac16 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:15.381211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ee9e0-bd22-40fe-b270-b74fce066fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.381878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ee9e0-bd22-40fe-b270-b74fce066fff None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.382160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ee9e0-bd22-40fe-b270-b74fce066fff None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.382429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ee9e0-bd22-40fe-b270-b74fce066fff None admin] PATH_INFO: `/v3/users/0f420bf7c9bb46ff8812779f03b5a892` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.383538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ee9e0-bd22-40fe-b270-b74fce066fff None admin] RBAC: Authorizing `identity:get_user(user_id=0f420bf7c9bb46ff8812779f03b5a892)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:15.385215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ee9e0-bd22-40fe-b270-b74fce066fff None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:15.386868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 546/1098] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:15 2022] GET /identity/v3/users/0f420bf7c9bb46ff8812779f03b5a892 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:15.392811 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ee9e0-bd22-40fe-b270-b74fce066fff None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-8b222cda-a707-423d-a020-ae499e36ac16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:40:15.396297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:15.397277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:40:15.398705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:40:15.398705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:40:15.398705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:15.398705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 553/1099] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:40:15.403060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b222cda-a707-423d-a020-ae499e36ac16 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:15.406219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:15.425640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:15.494322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ee9e0-bd22-40fe-b270-b74fce066fff None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.503205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b222cda-a707-423d-a020-ae499e36ac16 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.530210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6284c-3c91-48f0-9007-58fb13701730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.530821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6284c-3c91-48f0-9007-58fb13701730 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.531124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6284c-3c91-48f0-9007-58fb13701730 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.531387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6284c-3c91-48f0-9007-58fb13701730 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.537085 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc070793-da68-400d-99fd-510a3b20aff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.537713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070793-da68-400d-99fd-510a3b20aff8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.537994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070793-da68-400d-99fd-510a3b20aff8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.538268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070793-da68-400d-99fd-510a3b20aff8 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.538912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc070793-da68-400d-99fd-510a3b20aff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:15.539218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc070793-da68-400d-99fd-510a3b20aff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:15.540591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc070793-da68-400d-99fd-510a3b20aff8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:15.552528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 554/1100] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:15.558043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc070793-da68-400d-99fd-510a3b20aff8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:15.596773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6284c-3c91-48f0-9007-58fb13701730 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:15.598257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6284c-3c91-48f0-9007-58fb13701730 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:15.608851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:15.637854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:15.656930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc070793-da68-400d-99fd-510a3b20aff8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.689978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061683a1-aba7-443e-b75c-891796a41e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.690588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061683a1-aba7-443e-b75c-891796a41e16 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.690863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061683a1-aba7-443e-b75c-891796a41e16 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.691653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061683a1-aba7-443e-b75c-891796a41e16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.697254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 547/1101] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 14:40:15.698502 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061683a1-aba7-443e-b75c-891796a41e16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:15.699930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061683a1-aba7-443e-b75c-891796a41e16 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:15.703144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6284c-3c91-48f0-9007-58fb13701730 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-061683a1-aba7-443e-b75c-891796a41e16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:40:15.711096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:15.712391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:15.713287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:40:15.713287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:40:15.713287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:40:15.713287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:40:15.713287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:40:15.713287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:40:15.713287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:40:15.713287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:15.713287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 555/1102] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:40:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:40:15.717711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061683a1-aba7-443e-b75c-891796a41e16 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:15.801163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6284c-3c91-48f0-9007-58fb13701730 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.812322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:15.822736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061683a1-aba7-443e-b75c-891796a41e16 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.836762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fdad5-1b74-440c-aa45-298527b5bc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.837392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fdad5-1b74-440c-aa45-298527b5bc40 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.837698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fdad5-1b74-440c-aa45-298527b5bc40 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.837972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fdad5-1b74-440c-aa45-298527b5bc40 None admin] PATH_INFO: `/v3/projects/50363ca61fb446ccb560ed0bad95984d` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.839081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fdad5-1b74-440c-aa45-298527b5bc40 None admin] RBAC: Authorizing `identity:get_project(project_id=50363ca61fb446ccb560ed0bad95984d)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:15.840507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fdad5-1b74-440c-aa45-298527b5bc40 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:15.842172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 548/1103] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:40:15 2022] GET /identity/v3/projects/50363ca61fb446ccb560ed0bad95984d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:15.848795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fdad5-1b74-440c-aa45-298527b5bc40 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:15.858039 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaff48e-a0d9-4ae0-8df5-2baa364cbbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.858640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaff48e-a0d9-4ae0-8df5-2baa364cbbb2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.858914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaff48e-a0d9-4ae0-8df5-2baa364cbbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.859179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaff48e-a0d9-4ae0-8df5-2baa364cbbb2 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.859856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaff48e-a0d9-4ae0-8df5-2baa364cbbb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:15.860154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaff48e-a0d9-4ae0-8df5-2baa364cbbb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:15.861588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaff48e-a0d9-4ae0-8df5-2baa364cbbb2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:15.871785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:15.875208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 556/1104] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:40:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:15.881087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaff48e-a0d9-4ae0-8df5-2baa364cbbb2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:15.952766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fdad5-1b74-440c-aa45-298527b5bc40 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.977086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaff48e-a0d9-4ae0-8df5-2baa364cbbb2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:15.986408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e644c-ad7f-4486-b424-6acaa3c4ef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:15.987016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e644c-ad7f-4486-b424-6acaa3c4ef3a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:15.987298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e644c-ad7f-4486-b424-6acaa3c4ef3a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:15.987825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e644c-ad7f-4486-b424-6acaa3c4ef3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:15.988553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e644c-ad7f-4486-b424-6acaa3c4ef3a None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=50363ca61fb446ccb560ed0bad95984d, user.id=0f420bf7c9bb46ff8812779f03b5a892) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:15.988863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e644c-ad7f-4486-b424-6acaa3c4ef3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:15.990320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e644c-ad7f-4486-b424-6acaa3c4ef3a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:15.996161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 549/1105] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:40:15 2022] GET /identity/v3/role_assignments?user.id=0f420bf7c9bb46ff8812779f03b5a892&scope.project.id=50363ca61fb446ccb560ed0bad95984d&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:16.011764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012e256-718d-4303-a74b-76e13cf667ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:16.012384 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e256-718d-4303-a74b-76e13cf667ae None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:16.012658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e256-718d-4303-a74b-76e13cf667ae None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:16.012954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e256-718d-4303-a74b-76e13cf667ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:16.013735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012e256-718d-4303-a74b-76e13cf667ae None admin] RBAC: Adding query filter params (role.id=662ff20775c14ad69b5a84324701fcf1, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=1a4a95ae5ee84f259ffcd7528b0fa505) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:16.014043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012e256-718d-4303-a74b-76e13cf667ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:16.015519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012e256-718d-4303-a74b-76e13cf667ae None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:16.022618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 557/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:40:15 2022] GET /identity/v3/role_assignments?user.id=1a4a95ae5ee84f259ffcd7528b0fa505&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=662ff20775c14ad69b5a84324701fcf1 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:18.171564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6d41-cf2f-4810-9633-d0cd30eb6b57 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:18.172119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6d41-cf2f-4810-9633-d0cd30eb6b57 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:18.172119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6d41-cf2f-4810-9633-d0cd30eb6b57 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:18.173444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 550/1107] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:18.179183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8fbc1-b21a-41ff-958a-287949b2b827 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:18.179587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8fbc1-b21a-41ff-958a-287949b2b827 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:18.179733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8fbc1-b21a-41ff-958a-287949b2b827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:18.198314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e9e8fbc1-b21a-41ff-958a-287949b2b827 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:18.199498 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e9e8fbc1-b21a-41ff-958a-287949b2b827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:18.255891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:18.264364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 558/1108] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:18.533735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d6d41-cf2f-4810-9633-d0cd30eb6b57 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:18.576319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:18.631008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d6d41-cf2f-4810-9633-d0cd30eb6b57 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:18.663217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34763356-8087-44e1-b9a9-c6141f2ccac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:18.663828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34763356-8087-44e1-b9a9-c6141f2ccac7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:18.664106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34763356-8087-44e1-b9a9-c6141f2ccac7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:18.664376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34763356-8087-44e1-b9a9-c6141f2ccac7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:18.665461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34763356-8087-44e1-b9a9-c6141f2ccac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:18.666907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34763356-8087-44e1-b9a9-c6141f2ccac7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:18.668425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 551/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:18.674130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8fbc1-b21a-41ff-958a-287949b2b827 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:18.767827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8fbc1-b21a-41ff-958a-287949b2b827 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:18.799875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5368880-f643-49c4-84fc-c3480a3a27ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:18.800917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5368880-f643-49c4-84fc-c3480a3a27ff None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:18.801187 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5368880-f643-49c4-84fc-c3480a3a27ff None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:18.801449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5368880-f643-49c4-84fc-c3480a3a27ff None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:18.802107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5368880-f643-49c4-84fc-c3480a3a27ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:18.803507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5368880-f643-49c4-84fc-c3480a3a27ff None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:18.823947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 559/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:40:18 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:40:20.983957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ffeb-bb8b-4b71-b731-3e91abbe7d40 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:20.984484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ffeb-bb8b-4b71-b731-3e91abbe7d40 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:20.984484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ffeb-bb8b-4b71-b731-3e91abbe7d40 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:20.985859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 552/1111] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:20.991945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751859d3-ca6e-4484-87bc-04e01c795b46 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:20.992389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751859d3-ca6e-4484-87bc-04e01c795b46 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:20.992482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751859d3-ca6e-4484-87bc-04e01c795b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:21.010057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-751859d3-ca6e-4484-87bc-04e01c795b46 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:21.011224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-751859d3-ca6e-4484-87bc-04e01c795b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:21.055113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:21.075664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 560/1112] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:21.350613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550ffeb-bb8b-4b71-b731-3e91abbe7d40 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:21.446911 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550ffeb-bb8b-4b71-b731-3e91abbe7d40 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:21.458195 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:21.480807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674e147-abee-4608-bdf4-a5caf0e47b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:21.481448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674e147-abee-4608-bdf4-a5caf0e47b84 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:21.481759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674e147-abee-4608-bdf4-a5caf0e47b84 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:21.482031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674e147-abee-4608-bdf4-a5caf0e47b84 None admin] PATH_INFO: `/v3/roles/8cd21d388193490bac3aa386bcc18763` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:21.483454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674e147-abee-4608-bdf4-a5caf0e47b84 None admin] RBAC: Authorizing `identity:get_role(role_id=8cd21d388193490bac3aa386bcc18763)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:21.484853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674e147-abee-4608-bdf4-a5caf0e47b84 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:21.486237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 553/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:21 2022] GET /identity/v3/roles/8cd21d388193490bac3aa386bcc18763 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:21.492310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751859d3-ca6e-4484-87bc-04e01c795b46 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:21.586748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751859d3-ca6e-4484-87bc-04e01c795b46 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:21.618592 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004ba70-451d-46f2-90a0-4fbd5667d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:21.619201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004ba70-451d-46f2-90a0-4fbd5667d6b9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:21.619477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004ba70-451d-46f2-90a0-4fbd5667d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:21.619746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004ba70-451d-46f2-90a0-4fbd5667d6b9 None admin] PATH_INFO: `/v3/users/27ab1ec8dbd349d68e11dc24add38348` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:21.632579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004ba70-451d-46f2-90a0-4fbd5667d6b9 None admin] RBAC: Authorizing `identity:get_user(user_id=27ab1ec8dbd349d68e11dc24add38348)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:21.634039 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004ba70-451d-46f2-90a0-4fbd5667d6b9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:21.635588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 561/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:21 2022] GET /identity/v3/users/27ab1ec8dbd349d68e11dc24add38348 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:21.641167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674e147-abee-4608-bdf4-a5caf0e47b84 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:21.735105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674e147-abee-4608-bdf4-a5caf0e47b84 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:21.767406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f40a7-d2fd-48f6-b194-8a248c7d6bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:21.767988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f40a7-d2fd-48f6-b194-8a248c7d6bb6 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:21.768265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f40a7-d2fd-48f6-b194-8a248c7d6bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:21.768534 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f40a7-d2fd-48f6-b194-8a248c7d6bb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:21.811111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:21.832255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f40a7-d2fd-48f6-b194-8a248c7d6bb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:21.833726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f40a7-d2fd-48f6-b194-8a248c7d6bb6 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:21.925822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 554/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:40:21.931564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004ba70-451d-46f2-90a0-4fbd5667d6b9 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:21.970470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:22.027522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004ba70-451d-46f2-90a0-4fbd5667d6b9 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:22.059782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4fe18b-302b-4ffe-bcc8-17d5cb42f592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:22.060394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fe18b-302b-4ffe-bcc8-17d5cb42f592 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:22.060668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fe18b-302b-4ffe-bcc8-17d5cb42f592 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:22.060935 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fe18b-302b-4ffe-bcc8-17d5cb42f592 None admin] PATH_INFO: `/v3/projects/50363ca61fb446ccb560ed0bad95984d` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:22.062065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4fe18b-302b-4ffe-bcc8-17d5cb42f592 None admin] RBAC: Authorizing `identity:get_project(project_id=50363ca61fb446ccb560ed0bad95984d)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:22.063472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4fe18b-302b-4ffe-bcc8-17d5cb42f592 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:22.065049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 562/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:40:21 2022] GET /identity/v3/projects/50363ca61fb446ccb560ed0bad95984d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:22.071056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f40a7-d2fd-48f6-b194-8a248c7d6bb6 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:22.145679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:22.166990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f40a7-d2fd-48f6-b194-8a248c7d6bb6 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:22.199311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc52ee4-0cc3-4f90-80f9-f09418cd2cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:22.199852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc52ee4-0cc3-4f90-80f9-f09418cd2cbb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:22.200132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc52ee4-0cc3-4f90-80f9-f09418cd2cbb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:22.200406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc52ee4-0cc3-4f90-80f9-f09418cd2cbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:22.201127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc52ee4-0cc3-4f90-80f9-f09418cd2cbb None admin] RBAC: Adding query filter params (role.id=8cd21d388193490bac3aa386bcc18763, scope.project.id=50363ca61fb446ccb560ed0bad95984d, user.id=27ab1ec8dbd349d68e11dc24add38348) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:22.201435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc52ee4-0cc3-4f90-80f9-f09418cd2cbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:22.202880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc52ee4-0cc3-4f90-80f9-f09418cd2cbb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:22.208639 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 555/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:40:22 2022] GET /identity/v3/role_assignments?user.id=27ab1ec8dbd349d68e11dc24add38348&scope.project.id=50363ca61fb446ccb560ed0bad95984d&role.id=8cd21d388193490bac3aa386bcc18763 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:24.287574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf628b-66d4-4637-a9ca-e81faf055ab4 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:24.288108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf628b-66d4-4637-a9ca-e81faf055ab4 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:24.288219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf628b-66d4-4637-a9ca-e81faf055ab4 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:24.289520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 563/1118] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:24.294781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c3bb7-bb62-4705-9144-cecb29643fa6 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:24.295151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c3bb7-bb62-4705-9144-cecb29643fa6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:24.295316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c3bb7-bb62-4705-9144-cecb29643fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:24.312847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-105c3bb7-bb62-4705-9144-cecb29643fa6 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:24.314035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-105c3bb7-bb62-4705-9144-cecb29643fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:24.376906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 556/1119] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:24.654900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf628b-66d4-4637-a9ca-e81faf055ab4 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:24.751536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf628b-66d4-4637-a9ca-e81faf055ab4 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:24.759138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:24.785107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387ccb5-9dbb-4a4b-8ce7-e5d7ab0dea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:24.785745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387ccb5-9dbb-4a4b-8ce7-e5d7ab0dea31 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:24.786019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387ccb5-9dbb-4a4b-8ce7-e5d7ab0dea31 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:24.786291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387ccb5-9dbb-4a4b-8ce7-e5d7ab0dea31 None admin] PATH_INFO: `/v3/roles/8cd21d388193490bac3aa386bcc18763` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:24.787635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387ccb5-9dbb-4a4b-8ce7-e5d7ab0dea31 None admin] RBAC: Authorizing `identity:get_role(role_id=8cd21d388193490bac3aa386bcc18763)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:24.788981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387ccb5-9dbb-4a4b-8ce7-e5d7ab0dea31 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:24.790315 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 564/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:24 2022] GET /identity/v3/roles/8cd21d388193490bac3aa386bcc18763 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:24.796123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c3bb7-bb62-4705-9144-cecb29643fa6 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:24.863040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:24.891564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c3bb7-bb62-4705-9144-cecb29643fa6 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:24.923583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ea17f-76bc-4475-8939-77ed599d1151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:24.924144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ea17f-76bc-4475-8939-77ed599d1151 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:24.924423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ea17f-76bc-4475-8939-77ed599d1151 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:24.924698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ea17f-76bc-4475-8939-77ed599d1151 None admin] PATH_INFO: `/v3/users/27ab1ec8dbd349d68e11dc24add38348` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:24.926348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ea17f-76bc-4475-8939-77ed599d1151 None admin] RBAC: Authorizing `identity:get_user(user_id=27ab1ec8dbd349d68e11dc24add38348)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:24.927767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ea17f-76bc-4475-8939-77ed599d1151 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:24.929307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 557/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:24 2022] GET /identity/v3/users/27ab1ec8dbd349d68e11dc24add38348 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:24.935004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387ccb5-9dbb-4a4b-8ce7-e5d7ab0dea31 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:25.031307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387ccb5-9dbb-4a4b-8ce7-e5d7ab0dea31 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:25.063454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82a28c-095a-45bc-acc7-56ec62ebe5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:25.064064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82a28c-095a-45bc-acc7-56ec62ebe5cf None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:25.064341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82a28c-095a-45bc-acc7-56ec62ebe5cf None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:25.064609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82a28c-095a-45bc-acc7-56ec62ebe5cf None admin] PATH_INFO: `/v3/projects/50363ca61fb446ccb560ed0bad95984d` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:25.066333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82a28c-095a-45bc-acc7-56ec62ebe5cf None admin] RBAC: Authorizing `identity:get_project(project_id=50363ca61fb446ccb560ed0bad95984d)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:25.067758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82a28c-095a-45bc-acc7-56ec62ebe5cf None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:25.069364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 565/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:40:24 2022] GET /identity/v3/projects/50363ca61fb446ccb560ed0bad95984d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:25.075602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ea17f-76bc-4475-8939-77ed599d1151 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:25.172689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ea17f-76bc-4475-8939-77ed599d1151 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:25.206104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7b6ff-031d-4183-a6e4-71a3427d1ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:25.206689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7b6ff-031d-4183-a6e4-71a3427d1ed1 None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:25.206970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7b6ff-031d-4183-a6e4-71a3427d1ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:25.207241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7b6ff-031d-4183-a6e4-71a3427d1ed1 None admin] PATH_INFO: `/v3/projects/50363ca61fb446ccb560ed0bad95984d/users/27ab1ec8dbd349d68e11dc24add38348/roles/8cd21d388193490bac3aa386bcc18763` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:25.209336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7b6ff-031d-4183-a6e4-71a3427d1ed1 None admin] RBAC: Authorizing `identity:create_grant(project_id=50363ca61fb446ccb560ed0bad95984d, user_id=27ab1ec8dbd349d68e11dc24add38348, role_id=8cd21d388193490bac3aa386bcc18763)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:25.210789 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7b6ff-031d-4183-a6e4-71a3427d1ed1 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:25.219203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:25.224945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 558/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:40:25 2022] PUT /identity/v3/projects/50363ca61fb446ccb560ed0bad95984d/users/27ab1ec8dbd349d68e11dc24add38348/roles/8cd21d388193490bac3aa386bcc18763 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:40:27.340842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bbb98-5cd3-4401-bd22-5c8ff48cff68 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:27.341308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bbb98-5cd3-4401-bd22-5c8ff48cff68 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:27.341428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bbb98-5cd3-4401-bd22-5c8ff48cff68 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:27.342745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 566/1124] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:27.348018 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d3848-1528-41d1-ba29-e75e5fe3c03a None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:27.348418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d3848-1528-41d1-ba29-e75e5fe3c03a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:27.348568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d3848-1528-41d1-ba29-e75e5fe3c03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:27.366720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-546d3848-1528-41d1-ba29-e75e5fe3c03a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:27.367897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-546d3848-1528-41d1-ba29-e75e5fe3c03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:27.432180 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 559/1125] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:27.699334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bbb98-5cd3-4401-bd22-5c8ff48cff68 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:27.742093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:27.797197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bbb98-5cd3-4401-bd22-5c8ff48cff68 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:27.829661 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb44e73-ef3c-452f-97d8-e3c1d497b29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:27.830258 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44e73-ef3c-452f-97d8-e3c1d497b29d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:27.830535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44e73-ef3c-452f-97d8-e3c1d497b29d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:27.830827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44e73-ef3c-452f-97d8-e3c1d497b29d None admin] PATH_INFO: `/v3/roles/8cd21d388193490bac3aa386bcc18763` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:27.832174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb44e73-ef3c-452f-97d8-e3c1d497b29d None admin] RBAC: Authorizing `identity:get_role(role_id=8cd21d388193490bac3aa386bcc18763)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:27.833544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb44e73-ef3c-452f-97d8-e3c1d497b29d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:27.834821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 567/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:27 2022] GET /identity/v3/roles/8cd21d388193490bac3aa386bcc18763 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:27.840500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d3848-1528-41d1-ba29-e75e5fe3c03a None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:27.934574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d3848-1528-41d1-ba29-e75e5fe3c03a None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:27.961418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:27.967139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e55a9-7bef-4a76-b089-20d4a96c129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:27.967752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e55a9-7bef-4a76-b089-20d4a96c129d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:27.968025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e55a9-7bef-4a76-b089-20d4a96c129d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:27.968290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e55a9-7bef-4a76-b089-20d4a96c129d None admin] PATH_INFO: `/v3/users/27ab1ec8dbd349d68e11dc24add38348` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:27.969384 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e55a9-7bef-4a76-b089-20d4a96c129d None admin] RBAC: Authorizing `identity:get_user(user_id=27ab1ec8dbd349d68e11dc24add38348)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:27.970826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e55a9-7bef-4a76-b089-20d4a96c129d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:27.972414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 560/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:27 2022] GET /identity/v3/users/27ab1ec8dbd349d68e11dc24add38348 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:27.978091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb44e73-ef3c-452f-97d8-e3c1d497b29d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:28.072726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb44e73-ef3c-452f-97d8-e3c1d497b29d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:28.082432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:28.104715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c7011-bc4a-49ac-a922-40cf493898a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:28.105295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c7011-bc4a-49ac-a922-40cf493898a8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:28.105595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c7011-bc4a-49ac-a922-40cf493898a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:28.105865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c7011-bc4a-49ac-a922-40cf493898a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:28.170677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c7011-bc4a-49ac-a922-40cf493898a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:28.172115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c7011-bc4a-49ac-a922-40cf493898a8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:28.266276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 568/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 14:40:28.272554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e55a9-7bef-4a76-b089-20d4a96c129d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:28.368520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e55a9-7bef-4a76-b089-20d4a96c129d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:28.401817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f65b21-cec9-4922-b033-fb61d37ccf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:28.402438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f65b21-cec9-4922-b033-fb61d37ccf65 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:28.402718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f65b21-cec9-4922-b033-fb61d37ccf65 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:28.402993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f65b21-cec9-4922-b033-fb61d37ccf65 None admin] PATH_INFO: `/v3/projects/50363ca61fb446ccb560ed0bad95984d` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:28.404100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f65b21-cec9-4922-b033-fb61d37ccf65 None admin] RBAC: Authorizing `identity:get_project(project_id=50363ca61fb446ccb560ed0bad95984d)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:28.405553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f65b21-cec9-4922-b033-fb61d37ccf65 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:28.407157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 561/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:40:28 2022] GET /identity/v3/projects/50363ca61fb446ccb560ed0bad95984d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:28.413285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c7011-bc4a-49ac-a922-40cf493898a8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:28.424586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:28.509387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c7011-bc4a-49ac-a922-40cf493898a8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:28.542325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e4dd4-ca81-4718-ac44-c61504b8873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:28.542854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4dd4-ca81-4718-ac44-c61504b8873d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:28.543133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4dd4-ca81-4718-ac44-c61504b8873d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:28.543402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4dd4-ca81-4718-ac44-c61504b8873d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:28.544189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4dd4-ca81-4718-ac44-c61504b8873d None admin] RBAC: Adding query filter params (role.id=8cd21d388193490bac3aa386bcc18763, scope.project.id=50363ca61fb446ccb560ed0bad95984d, user.id=27ab1ec8dbd349d68e11dc24add38348) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:28.544498 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4dd4-ca81-4718-ac44-c61504b8873d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:28.545942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4dd4-ca81-4718-ac44-c61504b8873d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:28.551943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 569/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:40:28 2022] GET /identity/v3/role_assignments?user.id=27ab1ec8dbd349d68e11dc24add38348&scope.project.id=50363ca61fb446ccb560ed0bad95984d&role.id=8cd21d388193490bac3aa386bcc18763 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:30.688597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dcbdc-bfe3-4fc8-85c5-2a670e3cdcf9 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:30.689107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dcbdc-bfe3-4fc8-85c5-2a670e3cdcf9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:30.689211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dcbdc-bfe3-4fc8-85c5-2a670e3cdcf9 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:30.691005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 562/1131] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:30.696273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c0d01-fa2b-4efe-a7f2-ec43daa17518 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:30.696642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c0d01-fa2b-4efe-a7f2-ec43daa17518 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:30.696815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c0d01-fa2b-4efe-a7f2-ec43daa17518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:30.715033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-7f2c0d01-fa2b-4efe-a7f2-ec43daa17518 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:30.716204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7f2c0d01-fa2b-4efe-a7f2-ec43daa17518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:30.779725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 570/1132] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:31.043717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1dcbdc-bfe3-4fc8-85c5-2a670e3cdcf9 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:31.106817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:31.142875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1dcbdc-bfe3-4fc8-85c5-2a670e3cdcf9 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:31.175703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f2cc5-06d6-401e-846e-22c07fd359c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:31.176319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f2cc5-06d6-401e-846e-22c07fd359c3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:31.176597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f2cc5-06d6-401e-846e-22c07fd359c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:31.176869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f2cc5-06d6-401e-846e-22c07fd359c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:31.177990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f2cc5-06d6-401e-846e-22c07fd359c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:31.179445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f2cc5-06d6-401e-846e-22c07fd359c3 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:31.180975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 563/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:31.186686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c0d01-fa2b-4efe-a7f2-ec43daa17518 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:31.189712 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:31.283213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c0d01-fa2b-4efe-a7f2-ec43daa17518 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:31.316847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9546d-e8c4-4549-9d72-5ef4bd369087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:31.317386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9546d-e8c4-4549-9d72-5ef4bd369087 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:31.317701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9546d-e8c4-4549-9d72-5ef4bd369087 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:31.317972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9546d-e8c4-4549-9d72-5ef4bd369087 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:31.318615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9546d-e8c4-4549-9d72-5ef4bd369087 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:31.320041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9546d-e8c4-4549-9d72-5ef4bd369087 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:31.345785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 571/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:40:31 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:40:33.485919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194fe8-9e3c-45bb-974b-c4397c128ecf None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:33.486438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194fe8-9e3c-45bb-974b-c4397c128ecf None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:33.486525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194fe8-9e3c-45bb-974b-c4397c128ecf None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:33.488140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 564/1135] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:33.493370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0761d65-2fe3-42a6-8c08-e7fc55ccedbb None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:33.493759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0761d65-2fe3-42a6-8c08-e7fc55ccedbb None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:33.493950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0761d65-2fe3-42a6-8c08-e7fc55ccedbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:33.512181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-a0761d65-2fe3-42a6-8c08-e7fc55ccedbb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:33.513357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a0761d65-2fe3-42a6-8c08-e7fc55ccedbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:33.562192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:33.582041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 572/1136] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:33.847630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa194fe8-9e3c-45bb-974b-c4397c128ecf None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:33.944587 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa194fe8-9e3c-45bb-974b-c4397c128ecf None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:33.976958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff4f79-c4ad-448f-b0ef-f0e1ac6de02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:33.977594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff4f79-c4ad-448f-b0ef-f0e1ac6de02a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:33.977879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff4f79-c4ad-448f-b0ef-f0e1ac6de02a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:33.978153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff4f79-c4ad-448f-b0ef-f0e1ac6de02a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:33.979250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff4f79-c4ad-448f-b0ef-f0e1ac6de02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:33.980679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff4f79-c4ad-448f-b0ef-f0e1ac6de02a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:33.982235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 565/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:40:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:33.988309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0761d65-2fe3-42a6-8c08-e7fc55ccedbb None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:34.082483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0761d65-2fe3-42a6-8c08-e7fc55ccedbb None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:34.114598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70102df-12ca-40ea-9ca5-73b88519fca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:34.115121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70102df-12ca-40ea-9ca5-73b88519fca1 None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:34.115395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70102df-12ca-40ea-9ca5-73b88519fca1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:34.115666 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70102df-12ca-40ea-9ca5-73b88519fca1 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:34.116300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70102df-12ca-40ea-9ca5-73b88519fca1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:34.117741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70102df-12ca-40ea-9ca5-73b88519fca1 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:34.137973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 573/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:40:33 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:40:36.287202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e8334-ddab-44de-922b-da18cafbe0ff None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:36.287717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e8334-ddab-44de-922b-da18cafbe0ff None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:36.287717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e8334-ddab-44de-922b-da18cafbe0ff None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:36.289067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 566/1139] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:36.294276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cf938-5b0a-463c-99e4-d3baba2a24fd None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:36.294629 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cf938-5b0a-463c-99e4-d3baba2a24fd None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:36.294815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cf938-5b0a-463c-99e4-d3baba2a24fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:36.313529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-733cf938-5b0a-463c-99e4-d3baba2a24fd None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:36.315341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-733cf938-5b0a-463c-99e4-d3baba2a24fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:36.347953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:36.381145 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 574/1140] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:36.654372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e8334-ddab-44de-922b-da18cafbe0ff None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:36.657268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:36.754450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e8334-ddab-44de-922b-da18cafbe0ff None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:36.786637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbe985-7131-4e9f-86d4-68c19e83defc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:36.787249 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbe985-7131-4e9f-86d4-68c19e83defc None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:36.787530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbe985-7131-4e9f-86d4-68c19e83defc None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:36.787802 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbe985-7131-4e9f-86d4-68c19e83defc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:36.797424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbe985-7131-4e9f-86d4-68c19e83defc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-10bbe985-7131-4e9f-86d4-68c19e83defc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:40:36.802333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:36.803425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:36.804282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 567/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:40:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:36.809448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cf938-5b0a-463c-99e4-d3baba2a24fd None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:36.904755 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cf938-5b0a-463c-99e4-d3baba2a24fd None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:36.936999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8207e9-bff8-4ed2-823e-4bd092004b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:36.937620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8207e9-bff8-4ed2-823e-4bd092004b0d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:36.937900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8207e9-bff8-4ed2-823e-4bd092004b0d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:36.938169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8207e9-bff8-4ed2-823e-4bd092004b0d None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:36.938826 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8207e9-bff8-4ed2-823e-4bd092004b0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:36.939124 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8207e9-bff8-4ed2-823e-4bd092004b0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:36.940510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8207e9-bff8-4ed2-823e-4bd092004b0d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:36.948174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 575/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:40:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:36.954176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbe985-7131-4e9f-86d4-68c19e83defc None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:37.014369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:37.051672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbe985-7131-4e9f-86d4-68c19e83defc None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:37.083962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d93b3-5883-4c4b-a660-0076d961fd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:37.084579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d93b3-5883-4c4b-a660-0076d961fd74 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:37.084859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d93b3-5883-4c4b-a660-0076d961fd74 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:37.085134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d93b3-5883-4c4b-a660-0076d961fd74 None admin] PATH_INFO: `/v3/users/173985a7007640ecb164934a0a7edece` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:37.097558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d93b3-5883-4c4b-a660-0076d961fd74 None admin] RBAC: Authorizing `identity:get_user(user_id=173985a7007640ecb164934a0a7edece)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:37.098999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d93b3-5883-4c4b-a660-0076d961fd74 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:37.100558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 568/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:36 2022] GET /identity/v3/users/173985a7007640ecb164934a0a7edece => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:37.106263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8207e9-bff8-4ed2-823e-4bd092004b0d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:37.140198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:37.205204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8207e9-bff8-4ed2-823e-4bd092004b0d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:37.238140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fece70-f139-475b-b771-2061f33f3ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:37.238738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fece70-f139-475b-b771-2061f33f3ffa None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:37.239020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fece70-f139-475b-b771-2061f33f3ffa None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:37.239293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fece70-f139-475b-b771-2061f33f3ffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:37.303497 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fece70-f139-475b-b771-2061f33f3ffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:37.304949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fece70-f139-475b-b771-2061f33f3ffa None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:37.335688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:37.398972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 576/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:40:37.404985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d93b3-5883-4c4b-a660-0076d961fd74 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:37.500665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d93b3-5883-4c4b-a660-0076d961fd74 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:37.519484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:37.533093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d8d2d-56a2-4b97-a6c0-9de2642a578e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:37.534318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d8d2d-56a2-4b97-a6c0-9de2642a578e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:37.534598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d8d2d-56a2-4b97-a6c0-9de2642a578e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:37.534871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d8d2d-56a2-4b97-a6c0-9de2642a578e None admin] PATH_INFO: `/v3/projects/097f38dfe3964d9391ae3a87bc8385e6` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:37.535983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d8d2d-56a2-4b97-a6c0-9de2642a578e None admin] RBAC: Authorizing `identity:get_project(project_id=097f38dfe3964d9391ae3a87bc8385e6)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:37.537421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d8d2d-56a2-4b97-a6c0-9de2642a578e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:37.539045 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 569/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:40:37 2022] GET /identity/v3/projects/097f38dfe3964d9391ae3a87bc8385e6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:37.545010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fece70-f139-475b-b771-2061f33f3ffa None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:37.639833 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fece70-f139-475b-b771-2061f33f3ffa None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:37.673007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521eccca-af06-4825-bc7b-5d09ad03c39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:37.673618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eccca-af06-4825-bc7b-5d09ad03c39d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:37.673901 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eccca-af06-4825-bc7b-5d09ad03c39d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:37.674209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eccca-af06-4825-bc7b-5d09ad03c39d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:37.674919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521eccca-af06-4825-bc7b-5d09ad03c39d None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=097f38dfe3964d9391ae3a87bc8385e6, user.id=173985a7007640ecb164934a0a7edece) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:37.675225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521eccca-af06-4825-bc7b-5d09ad03c39d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:37.676640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521eccca-af06-4825-bc7b-5d09ad03c39d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:37.678395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:37.682841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 577/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:40:37 2022] GET /identity/v3/role_assignments?user.id=173985a7007640ecb164934a0a7edece&scope.project.id=097f38dfe3964d9391ae3a87bc8385e6&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:39.786012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305200b-da3a-47dd-8944-00ca24b050b4 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:39.786556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305200b-da3a-47dd-8944-00ca24b050b4 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:39.786556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305200b-da3a-47dd-8944-00ca24b050b4 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:39.787872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 570/1147] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:39.793154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48bd59-7b17-4b38-b59e-d0a26a3fc188 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:39.793600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48bd59-7b17-4b38-b59e-d0a26a3fc188 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:39.793695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48bd59-7b17-4b38-b59e-d0a26a3fc188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:39.811760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5c48bd59-7b17-4b38-b59e-d0a26a3fc188 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:39.812917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5c48bd59-7b17-4b38-b59e-d0a26a3fc188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:39.876376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 578/1148] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:40:40.146001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305200b-da3a-47dd-8944-00ca24b050b4 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:40.242522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305200b-da3a-47dd-8944-00ca24b050b4 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:40.274505 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af3d3b-ba34-45a2-9f47-0b752ec11c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:40.275112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af3d3b-ba34-45a2-9f47-0b752ec11c98 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:40.275398 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af3d3b-ba34-45a2-9f47-0b752ec11c98 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:40.275667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af3d3b-ba34-45a2-9f47-0b752ec11c98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:40.285974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af3d3b-ba34-45a2-9f47-0b752ec11c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-68af3d3b-ba34-45a2-9f47-0b752ec11c98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:40:40.290679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:40.291833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:40.292655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 571/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:40:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:40.297700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48bd59-7b17-4b38-b59e-d0a26a3fc188 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:40.392336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48bd59-7b17-4b38-b59e-d0a26a3fc188 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:40.417759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:40.426427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079d132-9b76-4057-868a-574c92e20722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:40.427019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079d132-9b76-4057-868a-574c92e20722 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:40.427300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079d132-9b76-4057-868a-574c92e20722 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:40.427572 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079d132-9b76-4057-868a-574c92e20722 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:40.428233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079d132-9b76-4057-868a-574c92e20722 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:40.428534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079d132-9b76-4057-868a-574c92e20722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:40.429933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079d132-9b76-4057-868a-574c92e20722 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:40.437879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 579/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:40:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:40.443537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af3d3b-ba34-45a2-9f47-0b752ec11c98 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:40.491595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:40.539751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af3d3b-ba34-45a2-9f47-0b752ec11c98 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:40.572430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d72f7-af23-4484-93d6-0cf6e95d3c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:40.573031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d72f7-af23-4484-93d6-0cf6e95d3c38 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:40.573307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d72f7-af23-4484-93d6-0cf6e95d3c38 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:40.573620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d72f7-af23-4484-93d6-0cf6e95d3c38 None admin] PATH_INFO: `/v3/users/173985a7007640ecb164934a0a7edece` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:40.575658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d72f7-af23-4484-93d6-0cf6e95d3c38 None admin] RBAC: Authorizing `identity:get_user(user_id=173985a7007640ecb164934a0a7edece)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:40.577068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d72f7-af23-4484-93d6-0cf6e95d3c38 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:40.578627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 572/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:40 2022] GET /identity/v3/users/173985a7007640ecb164934a0a7edece => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:40.584743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079d132-9b76-4057-868a-574c92e20722 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:40.681078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079d132-9b76-4057-868a-574c92e20722 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:40.712799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e1f20-564f-4920-adc4-e6cee192bd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:40.713410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e1f20-564f-4920-adc4-e6cee192bd66 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:40.713715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e1f20-564f-4920-adc4-e6cee192bd66 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:40.713988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e1f20-564f-4920-adc4-e6cee192bd66 None admin] PATH_INFO: `/v3/projects/097f38dfe3964d9391ae3a87bc8385e6` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:40.715082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e1f20-564f-4920-adc4-e6cee192bd66 None admin] RBAC: Authorizing `identity:get_project(project_id=097f38dfe3964d9391ae3a87bc8385e6)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:40.716502 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e1f20-564f-4920-adc4-e6cee192bd66 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:40.718114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 580/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:40:40 2022] GET /identity/v3/projects/097f38dfe3964d9391ae3a87bc8385e6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:40.724448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d72f7-af23-4484-93d6-0cf6e95d3c38 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:40.819056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d72f7-af23-4484-93d6-0cf6e95d3c38 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:40.835094 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:40.852993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae44d20-32cd-431b-ac81-9c8430f036f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:40.853662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44d20-32cd-431b-ac81-9c8430f036f0 None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:40.853947 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44d20-32cd-431b-ac81-9c8430f036f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:40.854222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44d20-32cd-431b-ac81-9c8430f036f0 None admin] PATH_INFO: `/v3/projects/097f38dfe3964d9391ae3a87bc8385e6/users/173985a7007640ecb164934a0a7edece/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:40.856099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae44d20-32cd-431b-ac81-9c8430f036f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=097f38dfe3964d9391ae3a87bc8385e6, user_id=173985a7007640ecb164934a0a7edece, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:40.857541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae44d20-32cd-431b-ac81-9c8430f036f0 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:40.870393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 573/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:40:40 2022] PUT /identity/v3/projects/097f38dfe3964d9391ae3a87bc8385e6/users/173985a7007640ecb164934a0a7edece/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:40:42.952241 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf5aff-5787-4b4e-b9cb-9402939e1b4f None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:42.952775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf5aff-5787-4b4e-b9cb-9402939e1b4f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:42.952775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf5aff-5787-4b4e-b9cb-9402939e1b4f None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:42.954135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 581/1154] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:42.959885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d1d77-b606-47f8-b0fe-e064b4d249e1 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:42.960254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d1d77-b606-47f8-b0fe-e064b4d249e1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:42.960440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d1d77-b606-47f8-b0fe-e064b4d249e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:42.979672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-881d1d77-b606-47f8-b0fe-e064b4d249e1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:42.981111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-881d1d77-b606-47f8-b0fe-e064b4d249e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:43.055046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 574/1155] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:43.364822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf5aff-5787-4b4e-b9cb-9402939e1b4f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:43.422188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:43.464262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf5aff-5787-4b4e-b9cb-9402939e1b4f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:43.496493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a64b5c-34e5-4f90-96c1-c0b9d72af9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:43.497106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a64b5c-34e5-4f90-96c1-c0b9d72af9df None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:43.497386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a64b5c-34e5-4f90-96c1-c0b9d72af9df None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:43.497813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a64b5c-34e5-4f90-96c1-c0b9d72af9df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:43.508098 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a64b5c-34e5-4f90-96c1-c0b9d72af9df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-61a64b5c-34e5-4f90-96c1-c0b9d72af9df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:40:43.512902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:43.514005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:43.514875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 582/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:43.520575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d1d77-b606-47f8-b0fe-e064b4d249e1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:43.616626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d1d77-b606-47f8-b0fe-e064b4d249e1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:43.623459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:43.650883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad386b4-f2a5-41f2-b1a2-f5f5ddf3d769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:43.651477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad386b4-f2a5-41f2-b1a2-f5f5ddf3d769 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:43.651755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad386b4-f2a5-41f2-b1a2-f5f5ddf3d769 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:43.652024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad386b4-f2a5-41f2-b1a2-f5f5ddf3d769 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:43.652684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad386b4-f2a5-41f2-b1a2-f5f5ddf3d769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:43.652986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad386b4-f2a5-41f2-b1a2-f5f5ddf3d769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:43.654361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad386b4-f2a5-41f2-b1a2-f5f5ddf3d769 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:43.662085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 575/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:43.668027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a64b5c-34e5-4f90-96c1-c0b9d72af9df None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:43.764720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a64b5c-34e5-4f90-96c1-c0b9d72af9df None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:43.780993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:43.797234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b35ce6-0881-400c-946d-029c6936d4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:43.797882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b35ce6-0881-400c-946d-029c6936d4aa None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:43.798158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b35ce6-0881-400c-946d-029c6936d4aa None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:43.798426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b35ce6-0881-400c-946d-029c6936d4aa None admin] PATH_INFO: `/v3/users/173985a7007640ecb164934a0a7edece` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:43.800018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b35ce6-0881-400c-946d-029c6936d4aa None admin] RBAC: Authorizing `identity:get_user(user_id=173985a7007640ecb164934a0a7edece)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:43.801437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b35ce6-0881-400c-946d-029c6936d4aa None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:43.803006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 583/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:43 2022] GET /identity/v3/users/173985a7007640ecb164934a0a7edece => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:43.808597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad386b4-f2a5-41f2-b1a2-f5f5ddf3d769 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:43.903089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad386b4-f2a5-41f2-b1a2-f5f5ddf3d769 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:43.934844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c94fb4-c203-41c9-8c89-dc47d99e4612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:43.935426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94fb4-c203-41c9-8c89-dc47d99e4612 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:43.935703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94fb4-c203-41c9-8c89-dc47d99e4612 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:43.935971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94fb4-c203-41c9-8c89-dc47d99e4612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:43.968877 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:44.000142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c94fb4-c203-41c9-8c89-dc47d99e4612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:44.001599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c94fb4-c203-41c9-8c89-dc47d99e4612 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:44.094602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 576/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:40:44.100349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b35ce6-0881-400c-946d-029c6936d4aa None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:44.196067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b35ce6-0881-400c-946d-029c6936d4aa None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:44.227747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb08a2d-863f-44f3-b370-d9e952c60f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:44.228350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb08a2d-863f-44f3-b370-d9e952c60f00 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:44.228626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb08a2d-863f-44f3-b370-d9e952c60f00 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:44.228896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb08a2d-863f-44f3-b370-d9e952c60f00 None admin] PATH_INFO: `/v3/projects/097f38dfe3964d9391ae3a87bc8385e6` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:44.230011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb08a2d-863f-44f3-b370-d9e952c60f00 None admin] RBAC: Authorizing `identity:get_project(project_id=097f38dfe3964d9391ae3a87bc8385e6)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:44.231445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb08a2d-863f-44f3-b370-d9e952c60f00 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:44.233012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 584/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:40:44 2022] GET /identity/v3/projects/097f38dfe3964d9391ae3a87bc8385e6 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:44.239135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c94fb4-c203-41c9-8c89-dc47d99e4612 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:44.302567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:44.334192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c94fb4-c203-41c9-8c89-dc47d99e4612 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:44.366266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38db4d3-9a04-40c6-a97d-f6673e7007cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:44.366852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38db4d3-9a04-40c6-a97d-f6673e7007cc None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:44.367127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38db4d3-9a04-40c6-a97d-f6673e7007cc None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:44.367394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38db4d3-9a04-40c6-a97d-f6673e7007cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:44.368108 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38db4d3-9a04-40c6-a97d-f6673e7007cc None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=097f38dfe3964d9391ae3a87bc8385e6, user.id=173985a7007640ecb164934a0a7edece) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:44.368410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38db4d3-9a04-40c6-a97d-f6673e7007cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:44.369842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38db4d3-9a04-40c6-a97d-f6673e7007cc None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:44.375360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 577/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:40:44 2022] GET /identity/v3/role_assignments?user.id=173985a7007640ecb164934a0a7edece&scope.project.id=097f38dfe3964d9391ae3a87bc8385e6&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:46.527512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d7a12-a070-47a2-a14d-c9c4b3c6f133 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:46.528092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d7a12-a070-47a2-a14d-c9c4b3c6f133 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:46.528092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d7a12-a070-47a2-a14d-c9c4b3c6f133 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:46.529384 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 585/1162] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:46.535609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dad85-7eaa-4050-8987-37d710fabf09 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:46.536048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dad85-7eaa-4050-8987-37d710fabf09 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:46.536317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dad85-7eaa-4050-8987-37d710fabf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:46.554400 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ef5dad85-7eaa-4050-8987-37d710fabf09 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:46.555500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ef5dad85-7eaa-4050-8987-37d710fabf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:46.619125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 578/1163] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:46.882505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d7a12-a070-47a2-a14d-c9c4b3c6f133 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:46.937201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:46.982367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d7a12-a070-47a2-a14d-c9c4b3c6f133 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:47.015127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15110138-d38d-4eff-a655-d47be8a96772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:47.015720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15110138-d38d-4eff-a655-d47be8a96772 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:47.015997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15110138-d38d-4eff-a655-d47be8a96772 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:47.016265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15110138-d38d-4eff-a655-d47be8a96772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:47.080711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15110138-d38d-4eff-a655-d47be8a96772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:47.082189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15110138-d38d-4eff-a655-d47be8a96772 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:47.133657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:47.176818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 586/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:40:47.182737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dad85-7eaa-4050-8987-37d710fabf09 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:47.242365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:47.280005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dad85-7eaa-4050-8987-37d710fabf09 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:47.312843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3b7cf-ae21-438b-964e-bf8a703edfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:47.313466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3b7cf-ae21-438b-964e-bf8a703edfeb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:47.313773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3b7cf-ae21-438b-964e-bf8a703edfeb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:47.314665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3b7cf-ae21-438b-964e-bf8a703edfeb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:47.321058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3b7cf-ae21-438b-964e-bf8a703edfeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:47.322687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3b7cf-ae21-438b-964e-bf8a703edfeb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-80d3b7cf-ae21-438b-964e-bf8a703edfeb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:47.334019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:47.335299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:40:47.336500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 579/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 3 14:40:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:47.341065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15110138-d38d-4eff-a655-d47be8a96772 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:47.435663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15110138-d38d-4eff-a655-d47be8a96772 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:47.468109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae7cfc-0519-4f5c-b04e-03c8eb206b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:47.468697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae7cfc-0519-4f5c-b04e-03c8eb206b4d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:47.468974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae7cfc-0519-4f5c-b04e-03c8eb206b4d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:47.469243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae7cfc-0519-4f5c-b04e-03c8eb206b4d None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:47.469905 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae7cfc-0519-4f5c-b04e-03c8eb206b4d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:47.470205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae7cfc-0519-4f5c-b04e-03c8eb206b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:47.471593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae7cfc-0519-4f5c-b04e-03c8eb206b4d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:47.478649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 587/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 3 14:40:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:49.640670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3520e67-4c80-4789-8096-687e6b5b535c None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:49.641186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3520e67-4c80-4789-8096-687e6b5b535c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:49.641186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3520e67-4c80-4789-8096-687e6b5b535c None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:49.642539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 580/1167] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:49.647616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71295ca4-3555-45ec-b862-15449d2d9312 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:49.648041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71295ca4-3555-45ec-b862-15449d2d9312 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:49.648163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71295ca4-3555-45ec-b862-15449d2d9312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:49.665817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-71295ca4-3555-45ec-b862-15449d2d9312 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:49.666990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-71295ca4-3555-45ec-b862-15449d2d9312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:49.680149 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:49.731113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 588/1168] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:49.996523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3520e67-4c80-4789-8096-687e6b5b535c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:50.093808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3520e67-4c80-4789-8096-687e6b5b535c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:50.115733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:50.127675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c32d1-6da1-49ec-a92d-733f413dba75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:50.128275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c32d1-6da1-49ec-a92d-733f413dba75 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:50.128556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c32d1-6da1-49ec-a92d-733f413dba75 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:50.128885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c32d1-6da1-49ec-a92d-733f413dba75 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:50.129511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c32d1-6da1-49ec-a92d-733f413dba75 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:50.130893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c32d1-6da1-49ec-a92d-733f413dba75 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:50.146156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 581/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Mon Oct 3 14:40:49 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:40:52.276953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f64c3d-47f6-4fd5-9b19-2ea9b6de3c90 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:52.277480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f64c3d-47f6-4fd5-9b19-2ea9b6de3c90 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:52.277480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f64c3d-47f6-4fd5-9b19-2ea9b6de3c90 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:52.278958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 589/1170] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:52.284422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663ac1d-a2c5-4e9e-9dad-3b7b1aeb5fc1 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:52.284829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663ac1d-a2c5-4e9e-9dad-3b7b1aeb5fc1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:52.284963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663ac1d-a2c5-4e9e-9dad-3b7b1aeb5fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:52.302477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-4663ac1d-a2c5-4e9e-9dad-3b7b1aeb5fc1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:52.303656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4663ac1d-a2c5-4e9e-9dad-3b7b1aeb5fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:52.366685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 582/1171] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:52.638882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f64c3d-47f6-4fd5-9b19-2ea9b6de3c90 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:52.739635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f64c3d-47f6-4fd5-9b19-2ea9b6de3c90 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:52.772358 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba95a0-a077-4641-a3f7-ed1b4f966793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:52.772967 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba95a0-a077-4641-a3f7-ed1b4f966793 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:52.773242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba95a0-a077-4641-a3f7-ed1b4f966793 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:52.773530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba95a0-a077-4641-a3f7-ed1b4f966793 None admin] PATH_INFO: `/v3/domains/1f0a42673f8c4ae0a8e2ad42c77297e8` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:52.783451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba95a0-a077-4641-a3f7-ed1b4f966793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f0a42673f8c4ae0a8e2ad42c77297e8)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:52.784876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba95a0-a077-4641-a3f7-ed1b4f966793 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:52.786469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 590/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Mon Oct 3 14:40:52 2022] GET /identity/v3/domains/1f0a42673f8c4ae0a8e2ad42c77297e8 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:40:52.792193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663ac1d-a2c5-4e9e-9dad-3b7b1aeb5fc1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:52.885637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663ac1d-a2c5-4e9e-9dad-3b7b1aeb5fc1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:52.898824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:52.919847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307ae12-736d-4e20-849b-fc18ca507107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:52.920433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307ae12-736d-4e20-849b-fc18ca507107 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:52.920709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307ae12-736d-4e20-849b-fc18ca507107 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:52.920975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307ae12-736d-4e20-849b-fc18ca507107 None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:52.921634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307ae12-736d-4e20-849b-fc18ca507107 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:52.923592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307ae12-736d-4e20-849b-fc18ca507107 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:52.947476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 583/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:40:52 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:40:55.104265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691fd3b-4640-4014-b11b-0495a79b62bc None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:55.104763 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691fd3b-4640-4014-b11b-0495a79b62bc None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:55.104763 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691fd3b-4640-4014-b11b-0495a79b62bc None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:55.106086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 591/1174] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:55.112124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0562f2-5b7d-4a28-9b16-8fd0140dca66 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:55.112554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0562f2-5b7d-4a28-9b16-8fd0140dca66 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:55.112652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0562f2-5b7d-4a28-9b16-8fd0140dca66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:55.131083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-5c0562f2-5b7d-4a28-9b16-8fd0140dca66 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:55.132269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5c0562f2-5b7d-4a28-9b16-8fd0140dca66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:55.195460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 584/1175] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:40:55.465877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691fd3b-4640-4014-b11b-0495a79b62bc None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:55.477007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:55.564299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691fd3b-4640-4014-b11b-0495a79b62bc None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:55.596848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b8997-fbdf-4109-a45c-ba7b172aee45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:55.597456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b8997-fbdf-4109-a45c-ba7b172aee45 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:55.597761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b8997-fbdf-4109-a45c-ba7b172aee45 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:55.598031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b8997-fbdf-4109-a45c-ba7b172aee45 None admin] PATH_INFO: `/v3/domains/1f0a42673f8c4ae0a8e2ad42c77297e8` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:55.599121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b8997-fbdf-4109-a45c-ba7b172aee45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f0a42673f8c4ae0a8e2ad42c77297e8)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:55.600538 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b8997-fbdf-4109-a45c-ba7b172aee45 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:55.602101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 592/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Mon Oct 3 14:40:55 2022] GET /identity/v3/domains/1f0a42673f8c4ae0a8e2ad42c77297e8 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:55.607782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0562f2-5b7d-4a28-9b16-8fd0140dca66 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:55.696426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:55.704656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0562f2-5b7d-4a28-9b16-8fd0140dca66 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:55.737646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1501f36-deeb-4c20-b30a-3e975a6f750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:55.738186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1501f36-deeb-4c20-b30a-3e975a6f750b None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:55.738465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1501f36-deeb-4c20-b30a-3e975a6f750b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:55.738738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1501f36-deeb-4c20-b30a-3e975a6f750b None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:55.739382 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1501f36-deeb-4c20-b30a-3e975a6f750b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:55.740800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1501f36-deeb-4c20-b30a-3e975a6f750b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:55.761089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 585/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:40:55 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:40:57.913627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8efa83-a2fe-42cb-8d62-4759eecb4d6b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:57.914177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8efa83-a2fe-42cb-8d62-4759eecb4d6b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:57.914177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8efa83-a2fe-42cb-8d62-4759eecb4d6b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:57.915507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 593/1178] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:40:57.920614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e643fba-cf5b-4338-82ef-63b88534924e None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:57.921031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e643fba-cf5b-4338-82ef-63b88534924e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:57.921143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e643fba-cf5b-4338-82ef-63b88534924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:57.938750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6e643fba-cf5b-4338-82ef-63b88534924e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:40:57.939917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6e643fba-cf5b-4338-82ef-63b88534924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:40:58.003806 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 586/1179] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:40:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:40:58.267366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8efa83-a2fe-42cb-8d62-4759eecb4d6b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:58.342448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:58.365964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8efa83-a2fe-42cb-8d62-4759eecb4d6b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:58.397459 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1692366-93a5-4011-8dd0-1ade8d28caa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:58.398100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1692366-93a5-4011-8dd0-1ade8d28caa9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:58.398372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1692366-93a5-4011-8dd0-1ade8d28caa9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:58.398634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1692366-93a5-4011-8dd0-1ade8d28caa9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:58.408683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1692366-93a5-4011-8dd0-1ade8d28caa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-c1692366-93a5-4011-8dd0-1ade8d28caa9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:40:58.413385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:40:58.414488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:40:58.415602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 594/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:40:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:40:58.420482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e643fba-cf5b-4338-82ef-63b88534924e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:58.496410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:58.515283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e643fba-cf5b-4338-82ef-63b88534924e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:58.547430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534b2bc-e01f-4753-86b6-bc3678e3cb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:58.548046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534b2bc-e01f-4753-86b6-bc3678e3cb3b None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:58.548326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534b2bc-e01f-4753-86b6-bc3678e3cb3b None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:58.548595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534b2bc-e01f-4753-86b6-bc3678e3cb3b None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:58.549255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534b2bc-e01f-4753-86b6-bc3678e3cb3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:58.549582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534b2bc-e01f-4753-86b6-bc3678e3cb3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:58.550949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534b2bc-e01f-4753-86b6-bc3678e3cb3b None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:58.558570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 587/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:40:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:58.564265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1692366-93a5-4011-8dd0-1ade8d28caa9 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:58.660320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1692366-93a5-4011-8dd0-1ade8d28caa9 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:58.691753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da1cbd-2e09-4127-ba78-b81fc4072b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:58.692352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da1cbd-2e09-4127-ba78-b81fc4072b94 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:58.692624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da1cbd-2e09-4127-ba78-b81fc4072b94 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:58.693094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da1cbd-2e09-4127-ba78-b81fc4072b94 None admin] PATH_INFO: `/v3/users/cb4872b16e5d422caf2faba3d042cc86` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:58.696883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:58.712512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da1cbd-2e09-4127-ba78-b81fc4072b94 None admin] RBAC: Authorizing `identity:get_user(user_id=cb4872b16e5d422caf2faba3d042cc86)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:58.714824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da1cbd-2e09-4127-ba78-b81fc4072b94 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:58.717258 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 595/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:40:58 2022] GET /identity/v3/users/cb4872b16e5d422caf2faba3d042cc86 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:58.723718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534b2bc-e01f-4753-86b6-bc3678e3cb3b None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:58.818101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534b2bc-e01f-4753-86b6-bc3678e3cb3b None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:58.850260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540ce97-64f6-4362-8c2a-1b4b23d44047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:58.850858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540ce97-64f6-4362-8c2a-1b4b23d44047 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:58.851136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540ce97-64f6-4362-8c2a-1b4b23d44047 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:58.851407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540ce97-64f6-4362-8c2a-1b4b23d44047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:58.862803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:58.916234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540ce97-64f6-4362-8c2a-1b4b23d44047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:58.917720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540ce97-64f6-4362-8c2a-1b4b23d44047 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:59.011665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 588/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:40:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:40:59.018291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da1cbd-2e09-4127-ba78-b81fc4072b94 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:59.117686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da1cbd-2e09-4127-ba78-b81fc4072b94 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:59.150872 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2aecc-c22c-4536-8311-37b7f109033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:59.151487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2aecc-c22c-4536-8311-37b7f109033a None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:59.151763 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2aecc-c22c-4536-8311-37b7f109033a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:59.152030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2aecc-c22c-4536-8311-37b7f109033a None admin] PATH_INFO: `/v3/projects/68ab584187134bcab2e4026e0b29ce40` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:59.153120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2aecc-c22c-4536-8311-37b7f109033a None admin] RBAC: Authorizing `identity:get_project(project_id=68ab584187134bcab2e4026e0b29ce40)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:59.154577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2aecc-c22c-4536-8311-37b7f109033a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:59.156175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 596/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:40:59 2022] GET /identity/v3/projects/68ab584187134bcab2e4026e0b29ce40 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:40:59.162478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540ce97-64f6-4362-8c2a-1b4b23d44047 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:40:59.202638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:40:59.256102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540ce97-64f6-4362-8c2a-1b4b23d44047 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:40:59.289092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8e33a-8ec5-404f-97ca-70dcdb0434a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:40:59.289886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8e33a-8ec5-404f-97ca-70dcdb0434a8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:40:59.290171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8e33a-8ec5-404f-97ca-70dcdb0434a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:40:59.290443 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8e33a-8ec5-404f-97ca-70dcdb0434a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:40:59.291178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8e33a-8ec5-404f-97ca-70dcdb0434a8 None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=68ab584187134bcab2e4026e0b29ce40, user.id=cb4872b16e5d422caf2faba3d042cc86) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:40:59.291487 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8e33a-8ec5-404f-97ca-70dcdb0434a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:40:59.292914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8e33a-8ec5-404f-97ca-70dcdb0434a8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:40:59.298664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 589/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:40:59 2022] GET /identity/v3/role_assignments?user.id=cb4872b16e5d422caf2faba3d042cc86&scope.project.id=68ab584187134bcab2e4026e0b29ce40&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:41:01.392095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44612a9-b7a3-4308-8a19-97cd28e271d8 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:01.392629 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44612a9-b7a3-4308-8a19-97cd28e271d8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:01.392629 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44612a9-b7a3-4308-8a19-97cd28e271d8 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:01.393994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 597/1186] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:41:01.399303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cf7f5-555d-4466-a033-5090f92523e5 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:01.399668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cf7f5-555d-4466-a033-5090f92523e5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:01.399855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cf7f5-555d-4466-a033-5090f92523e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:01.417932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-055cf7f5-555d-4466-a033-5090f92523e5 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:41:01.419111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-055cf7f5-555d-4466-a033-5090f92523e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:41:01.482102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 590/1187] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:41:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:41:01.753136 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44612a9-b7a3-4308-8a19-97cd28e271d8 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:01.813244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:41:01.851705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44612a9-b7a3-4308-8a19-97cd28e271d8 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:01.883756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3efed-e753-4f7d-ba40-fb91de0b05a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:01.884362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3efed-e753-4f7d-ba40-fb91de0b05a2 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:01.884639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3efed-e753-4f7d-ba40-fb91de0b05a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:01.884910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3efed-e753-4f7d-ba40-fb91de0b05a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:01.895077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3efed-e753-4f7d-ba40-fb91de0b05a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-7cb3efed-e753-4f7d-ba40-fb91de0b05a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:41:01.899776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:41:01.900821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:41:01.901869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 598/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:41:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:41:01.906764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cf7f5-555d-4466-a033-5090f92523e5 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:01.940316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:41:02.003494 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cf7f5-555d-4466-a033-5090f92523e5 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:02.036129 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a47c7c-6208-447a-b0be-c89dfdb96726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:02.036727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a47c7c-6208-447a-b0be-c89dfdb96726 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:02.037009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a47c7c-6208-447a-b0be-c89dfdb96726 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:02.037283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a47c7c-6208-447a-b0be-c89dfdb96726 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:02.037977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a47c7c-6208-447a-b0be-c89dfdb96726 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:41:02.038282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a47c7c-6208-447a-b0be-c89dfdb96726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:41:02.039647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a47c7c-6208-447a-b0be-c89dfdb96726 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:41:02.047351 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 591/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:41:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:41:02.053101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3efed-e753-4f7d-ba40-fb91de0b05a2 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:02.148763 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3efed-e753-4f7d-ba40-fb91de0b05a2 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:02.170002 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:41:02.181341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d07e0-cf08-499c-a457-cc0fde73c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:02.181976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d07e0-cf08-499c-a457-cc0fde73c382 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:02.182254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d07e0-cf08-499c-a457-cc0fde73c382 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:02.182524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d07e0-cf08-499c-a457-cc0fde73c382 None admin] PATH_INFO: `/v3/users/cb4872b16e5d422caf2faba3d042cc86` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:02.183617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d07e0-cf08-499c-a457-cc0fde73c382 None admin] RBAC: Authorizing `identity:get_user(user_id=cb4872b16e5d422caf2faba3d042cc86)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:41:02.185040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d07e0-cf08-499c-a457-cc0fde73c382 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:41:02.186607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 599/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:41:02 2022] GET /identity/v3/users/cb4872b16e5d422caf2faba3d042cc86 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:41:02.192676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a47c7c-6208-447a-b0be-c89dfdb96726 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:02.285618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:41:02.289797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a47c7c-6208-447a-b0be-c89dfdb96726 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:02.324600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a43b7-358b-4528-86ee-4e75c65b5a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:02.325229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a43b7-358b-4528-86ee-4e75c65b5a55 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:02.325522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a43b7-358b-4528-86ee-4e75c65b5a55 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:02.325806 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a43b7-358b-4528-86ee-4e75c65b5a55 None admin] PATH_INFO: `/v3/projects/68ab584187134bcab2e4026e0b29ce40` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:02.326904 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a43b7-358b-4528-86ee-4e75c65b5a55 None admin] RBAC: Authorizing `identity:get_project(project_id=68ab584187134bcab2e4026e0b29ce40)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:41:02.328330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a43b7-358b-4528-86ee-4e75c65b5a55 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:41:02.329983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 592/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:41:02 2022] GET /identity/v3/projects/68ab584187134bcab2e4026e0b29ce40 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:41:02.335919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d07e0-cf08-499c-a457-cc0fde73c382 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:02.431536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d07e0-cf08-499c-a457-cc0fde73c382 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:02.463211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619356e5-5642-430f-bc3b-0023258b3a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:02.463842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619356e5-5642-430f-bc3b-0023258b3a74 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:02.464116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619356e5-5642-430f-bc3b-0023258b3a74 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:02.464384 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619356e5-5642-430f-bc3b-0023258b3a74 None admin] PATH_INFO: `/v3/projects/68ab584187134bcab2e4026e0b29ce40/users/cb4872b16e5d422caf2faba3d042cc86/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:02.466236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619356e5-5642-430f-bc3b-0023258b3a74 None admin] RBAC: Authorizing `identity:create_grant(project_id=68ab584187134bcab2e4026e0b29ce40, user_id=cb4872b16e5d422caf2faba3d042cc86, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:41:02.467649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619356e5-5642-430f-bc3b-0023258b3a74 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:41:02.479779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 600/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:41:02 2022] PUT /identity/v3/projects/68ab584187134bcab2e4026e0b29ce40/users/cb4872b16e5d422caf2faba3d042cc86/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:41:04.592300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5fd6-683f-4588-bceb-86f20d4b3d6a None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:04.592794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5fd6-683f-4588-bceb-86f20d4b3d6a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:04.592794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5fd6-683f-4588-bceb-86f20d4b3d6a None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:04.594132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 593/1193] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:41:04.599433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76228-2069-438d-acc4-93717b0e963b None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:04.599925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76228-2069-438d-acc4-93717b0e963b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:04.600019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76228-2069-438d-acc4-93717b0e963b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:04.618438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-38e76228-2069-438d-acc4-93717b0e963b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:41:04.619639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-38e76228-2069-438d-acc4-93717b0e963b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:41:04.664444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:41:04.685468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 601/1194] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:41:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:41:04.956485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e5fd6-683f-4588-bceb-86f20d4b3d6a None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:05.053239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e5fd6-683f-4588-bceb-86f20d4b3d6a None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:05.085451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b1efb-6e22-4c63-bdb6-0d5c528ef725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:05.086099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b1efb-6e22-4c63-bdb6-0d5c528ef725 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:05.086377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b1efb-6e22-4c63-bdb6-0d5c528ef725 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:05.086648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b1efb-6e22-4c63-bdb6-0d5c528ef725 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:05.097365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b1efb-6e22-4c63-bdb6-0d5c528ef725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-e53b1efb-6e22-4c63-bdb6-0d5c528ef725 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:41:05.102061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:41:05.103146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:41:05.104353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 594/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:41:05.108917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e76228-2069-438d-acc4-93717b0e963b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:05.203202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e76228-2069-438d-acc4-93717b0e963b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:05.235780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f230f0-0b4c-4ba4-ab7a-a0b12917dbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:05.236587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f230f0-0b4c-4ba4-ab7a-a0b12917dbfe None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:05.236864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f230f0-0b4c-4ba4-ab7a-a0b12917dbfe None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:05.237134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f230f0-0b4c-4ba4-ab7a-a0b12917dbfe None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:05.237816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f230f0-0b4c-4ba4-ab7a-a0b12917dbfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:41:05.238117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f230f0-0b4c-4ba4-ab7a-a0b12917dbfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:41:05.239482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f230f0-0b4c-4ba4-ab7a-a0b12917dbfe None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:41:05.247151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 602/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:41:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:41:05.253288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b1efb-6e22-4c63-bdb6-0d5c528ef725 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:05.280332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:41:05.350844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b1efb-6e22-4c63-bdb6-0d5c528ef725 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:05.383582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef185a-e79c-4855-aeb2-b4893d6b2ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:05.384202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef185a-e79c-4855-aeb2-b4893d6b2ba9 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:05.384482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef185a-e79c-4855-aeb2-b4893d6b2ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:05.384755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef185a-e79c-4855-aeb2-b4893d6b2ba9 None admin] PATH_INFO: `/v3/users/cb4872b16e5d422caf2faba3d042cc86` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:05.385886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef185a-e79c-4855-aeb2-b4893d6b2ba9 None admin] RBAC: Authorizing `identity:get_user(user_id=cb4872b16e5d422caf2faba3d042cc86)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:41:05.387322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef185a-e79c-4855-aeb2-b4893d6b2ba9 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:41:05.388878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 595/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 3 14:41:05 2022] GET /identity/v3/users/cb4872b16e5d422caf2faba3d042cc86 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:41:05.394621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f230f0-0b4c-4ba4-ab7a-a0b12917dbfe None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:05.436239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:41:05.491392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f230f0-0b4c-4ba4-ab7a-a0b12917dbfe None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:05.522784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f99ba8-0541-45ff-bb69-4dd1621f1384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:05.523365 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f99ba8-0541-45ff-bb69-4dd1621f1384 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:05.523637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f99ba8-0541-45ff-bb69-4dd1621f1384 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:05.523902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f99ba8-0541-45ff-bb69-4dd1621f1384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:05.586836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f99ba8-0541-45ff-bb69-4dd1621f1384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:41:05.588265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f99ba8-0541-45ff-bb69-4dd1621f1384 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:41:05.628427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:41:05.683054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 603/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:41:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:41:05.688918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef185a-e79c-4855-aeb2-b4893d6b2ba9 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:05.777890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:41:05.785094 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef185a-e79c-4855-aeb2-b4893d6b2ba9 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:05.817014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00ba9c-df73-4ac6-ab34-3067036416e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:05.817657 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00ba9c-df73-4ac6-ab34-3067036416e6 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:05.817938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00ba9c-df73-4ac6-ab34-3067036416e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:05.818208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00ba9c-df73-4ac6-ab34-3067036416e6 None admin] PATH_INFO: `/v3/projects/68ab584187134bcab2e4026e0b29ce40` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:05.819315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00ba9c-df73-4ac6-ab34-3067036416e6 None admin] RBAC: Authorizing `identity:get_project(project_id=68ab584187134bcab2e4026e0b29ce40)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:41:05.820731 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00ba9c-df73-4ac6-ab34-3067036416e6 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:41:05.822396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 596/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 3 14:41:05 2022] GET /identity/v3/projects/68ab584187134bcab2e4026e0b29ce40 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:41:05.828311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f99ba8-0541-45ff-bb69-4dd1621f1384 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:41:05.923215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f99ba8-0541-45ff-bb69-4dd1621f1384 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:41:05.955309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950af740-82a9-47ea-8435-5090e36865d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:41:05.955891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950af740-82a9-47ea-8435-5090e36865d9 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:41:05.956167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950af740-82a9-47ea-8435-5090e36865d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:41:05.956434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950af740-82a9-47ea-8435-5090e36865d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:41:05.957138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950af740-82a9-47ea-8435-5090e36865d9 None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=68ab584187134bcab2e4026e0b29ce40, user.id=cb4872b16e5d422caf2faba3d042cc86) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:41:05.957439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950af740-82a9-47ea-8435-5090e36865d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:41:05.958879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950af740-82a9-47ea-8435-5090e36865d9 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:41:05.964424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 604/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:41:05 2022] GET /identity/v3/role_assignments?user.id=cb4872b16e5d422caf2faba3d042cc86&scope.project.id=68ab584187134bcab2e4026e0b29ce40&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:41:15.812994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=19 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:41:15.961926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:01.565624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea69a2-0f36-49f9-82b9-e7fd8df57f3e None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:01.566627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea69a2-0f36-49f9-82b9-e7fd8df57f3e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:01.566912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea69a2-0f36-49f9-82b9-e7fd8df57f3e None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:01.568344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 597/1201] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:01.574427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5681060-96f4-4fde-96f6-ef0cb9ae1b3e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:01.575927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5681060-96f4-4fde-96f6-ef0cb9ae1b3e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:01.576414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5681060-96f4-4fde-96f6-ef0cb9ae1b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:01.595631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b5681060-96f4-4fde-96f6-ef0cb9ae1b3e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:01.596899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b5681060-96f4-4fde-96f6-ef0cb9ae1b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:01.679932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 605/1202] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:42:01.992516 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea69a2-0f36-49f9-82b9-e7fd8df57f3e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:02.131678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea69a2-0f36-49f9-82b9-e7fd8df57f3e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:02.173342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cb180-7021-4bf6-8879-69caba83146a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:02.174225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cb180-7021-4bf6-8879-69caba83146a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:02.174509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cb180-7021-4bf6-8879-69caba83146a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:02.174785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cb180-7021-4bf6-8879-69caba83146a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:02.191579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cb180-7021-4bf6-8879-69caba83146a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:02.192261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cb180-7021-4bf6-8879-69caba83146a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-572cb180-7021-4bf6-8879-69caba83146a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:02.209944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:02.211058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:02.218032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:02.218032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:02.218032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:02.218032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 598/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:02.218284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5681060-96f4-4fde-96f6-ef0cb9ae1b3e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:02.347328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5681060-96f4-4fde-96f6-ef0cb9ae1b3e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:02.384655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:02.393387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409412f-2b84-4bf5-a711-e2ae1056bbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:02.394293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409412f-2b84-4bf5-a711-e2ae1056bbb1 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:02.394577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409412f-2b84-4bf5-a711-e2ae1056bbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:02.394852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409412f-2b84-4bf5-a711-e2ae1056bbb1 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:02.395504 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409412f-2b84-4bf5-a711-e2ae1056bbb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:02.395808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409412f-2b84-4bf5-a711-e2ae1056bbb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:02.397182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409412f-2b84-4bf5-a711-e2ae1056bbb1 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:02.410885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 606/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:02.417627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cb180-7021-4bf6-8879-69caba83146a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:02.492756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:02.566920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cb180-7021-4bf6-8879-69caba83146a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:02.617212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbe9c8-15ee-4951-990c-7b1ed5afc5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:02.618115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbe9c8-15ee-4951-990c-7b1ed5afc5b6 None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:02.618383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbe9c8-15ee-4951-990c-7b1ed5afc5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:02.618722 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbe9c8-15ee-4951-990c-7b1ed5afc5b6 None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:02.619485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbe9c8-15ee-4951-990c-7b1ed5afc5b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:02.621262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbe9c8-15ee-4951-990c-7b1ed5afc5b6 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:02.652724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 599/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 3 14:42:02 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:42:05.171029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32016a44-8199-448c-8c9d-f5a53ec2307b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:05.171584 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32016a44-8199-448c-8c9d-f5a53ec2307b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:05.171911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32016a44-8199-448c-8c9d-f5a53ec2307b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:05.173339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 607/1206] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:05.178969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5114cf-af32-4280-a5b2-dc8500731a8d None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:05.179403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5114cf-af32-4280-a5b2-dc8500731a8d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:05.179581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5114cf-af32-4280-a5b2-dc8500731a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:05.200092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-8b5114cf-af32-4280-a5b2-dc8500731a8d None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:05.201530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8b5114cf-af32-4280-a5b2-dc8500731a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:05.286597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 600/1207] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:42:05.590853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32016a44-8199-448c-8c9d-f5a53ec2307b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:05.719065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32016a44-8199-448c-8c9d-f5a53ec2307b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:05.760555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81f212-76b8-49d7-844a-61ca46c247d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:05.761245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81f212-76b8-49d7-844a-61ca46c247d4 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:05.761606 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81f212-76b8-49d7-844a-61ca46c247d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:05.761888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81f212-76b8-49d7-844a-61ca46c247d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:05.773040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81f212-76b8-49d7-844a-61ca46c247d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-af81f212-76b8-49d7-844a-61ca46c247d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:42:05.778774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:05.781105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:05.782010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 608/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:42:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:05.786668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5114cf-af32-4280-a5b2-dc8500731a8d None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:05.833140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:05.903684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5114cf-af32-4280-a5b2-dc8500731a8d None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:05.946054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f049512-90e3-44f2-bccb-1eb2feeeef7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:05.946736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f049512-90e3-44f2-bccb-1eb2feeeef7e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:05.947073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f049512-90e3-44f2-bccb-1eb2feeeef7e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:05.947377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f049512-90e3-44f2-bccb-1eb2feeeef7e None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:05.948150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f049512-90e3-44f2-bccb-1eb2feeeef7e None admin] RBAC: Adding query filter params (name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:05.948508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f049512-90e3-44f2-bccb-1eb2feeeef7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:05.950070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f049512-90e3-44f2-bccb-1eb2feeeef7e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:05.959381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 601/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:42:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:05.966493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81f212-76b8-49d7-844a-61ca46c247d4 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:06.022645 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:06.086256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81f212-76b8-49d7-844a-61ca46c247d4 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:06.128269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4473040-7329-4c4e-af05-c371d39802d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:06.128933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4473040-7329-4c4e-af05-c371d39802d7 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:06.129220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4473040-7329-4c4e-af05-c371d39802d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:06.129555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4473040-7329-4c4e-af05-c371d39802d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:06.142571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4473040-7329-4c4e-af05-c371d39802d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:06.144419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4473040-7329-4c4e-af05-c371d39802d7 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-b4473040-7329-4c4e-af05-c371d39802d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:06.163570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:06.164631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:06.166048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:06.166048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:06.166048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:06.166048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 609/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:06.171787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f049512-90e3-44f2-bccb-1eb2feeeef7e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:06.288511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:06.292000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f049512-90e3-44f2-bccb-1eb2feeeef7e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:06.335630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf3b65-402e-4905-907c-274ee61d228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:06.336367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf3b65-402e-4905-907c-274ee61d228a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:06.336653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf3b65-402e-4905-907c-274ee61d228a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:06.337205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf3b65-402e-4905-907c-274ee61d228a None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:06.338040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf3b65-402e-4905-907c-274ee61d228a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:06.338396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf3b65-402e-4905-907c-274ee61d228a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:06.340464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf3b65-402e-4905-907c-274ee61d228a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:06.362396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 602/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:06.364241 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4473040-7329-4c4e-af05-c371d39802d7 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:06.482485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4473040-7329-4c4e-af05-c371d39802d7 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:06.484669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:06.529725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c745b17-dbb3-4eb1-828b-7db07316efe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:06.530348 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c745b17-dbb3-4eb1-828b-7db07316efe8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:06.530622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c745b17-dbb3-4eb1-828b-7db07316efe8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:06.530887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c745b17-dbb3-4eb1-828b-7db07316efe8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:06.539000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c745b17-dbb3-4eb1-828b-7db07316efe8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:06.540450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c745b17-dbb3-4eb1-828b-7db07316efe8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-8c745b17-dbb3-4eb1-828b-7db07316efe8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:42:06.552162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:06.553470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:06.554794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:06.554794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:06.554794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:42:06.554794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:42:06.554794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:42:06.554794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:42:06.554794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:42:06.554794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:42:06.554794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:06.554794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:06.558486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 610/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:06.562788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf3b65-402e-4905-907c-274ee61d228a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:06.684837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf3b65-402e-4905-907c-274ee61d228a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:06.723313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971a380-ce2c-4af5-91f3-c8c3af18fe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:06.723914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971a380-ce2c-4af5-91f3-c8c3af18fe1c None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:06.724198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971a380-ce2c-4af5-91f3-c8c3af18fe1c None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:06.724465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971a380-ce2c-4af5-91f3-c8c3af18fe1c None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:06.725192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971a380-ce2c-4af5-91f3-c8c3af18fe1c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:06.725525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971a380-ce2c-4af5-91f3-c8c3af18fe1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:06.726938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971a380-ce2c-4af5-91f3-c8c3af18fe1c None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:06.746360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 603/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 3 14:42:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:06.752247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c745b17-dbb3-4eb1-828b-7db07316efe8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:06.871639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c745b17-dbb3-4eb1-828b-7db07316efe8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:06.915210 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492167c-543f-4591-9e9d-e0af2e9b29f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:06.915971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492167c-543f-4591-9e9d-e0af2e9b29f6 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:06.916313 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492167c-543f-4591-9e9d-e0af2e9b29f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:06.916622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492167c-543f-4591-9e9d-e0af2e9b29f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:06.950646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:06.999698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492167c-543f-4591-9e9d-e0af2e9b29f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:07.001516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492167c-543f-4591-9e9d-e0af2e9b29f6 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:07.148593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 611/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:42:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:42:07.155401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971a380-ce2c-4af5-91f3-c8c3af18fe1c None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:07.177883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:07.285959 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971a380-ce2c-4af5-91f3-c8c3af18fe1c None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:07.334474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fa68e-56ed-441b-8e11-4d4e8157e470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:07.335254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fa68e-56ed-441b-8e11-4d4e8157e470 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:07.335539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fa68e-56ed-441b-8e11-4d4e8157e470 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:07.336016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fa68e-56ed-441b-8e11-4d4e8157e470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:07.349706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0fa68e-56ed-441b-8e11-4d4e8157e470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:07.351717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0fa68e-56ed-441b-8e11-4d4e8157e470 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-0b0fa68e-56ed-441b-8e11-4d4e8157e470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:07.370748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:07.372173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:07.373622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:07.373622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:07.373622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:07.373622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 604/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:07.380298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492167c-543f-4591-9e9d-e0af2e9b29f6 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:07.442739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:07.489063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492167c-543f-4591-9e9d-e0af2e9b29f6 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:07.528668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17930226-47bc-447b-9f52-908ae750b235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:07.529277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930226-47bc-447b-9f52-908ae750b235 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:07.529614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930226-47bc-447b-9f52-908ae750b235 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:07.529857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930226-47bc-447b-9f52-908ae750b235 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:07.530495 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17930226-47bc-447b-9f52-908ae750b235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:07.530792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17930226-47bc-447b-9f52-908ae750b235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:07.532453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17930226-47bc-447b-9f52-908ae750b235 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:07.551307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 612/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:07.552762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fa68e-56ed-441b-8e11-4d4e8157e470 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:07.631324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:07.668917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fa68e-56ed-441b-8e11-4d4e8157e470 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:07.707365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97796b98-8dc2-44c9-9a9b-0ccee74beb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:07.708061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97796b98-8dc2-44c9-9a9b-0ccee74beb83 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:07.708387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97796b98-8dc2-44c9-9a9b-0ccee74beb83 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:07.708692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97796b98-8dc2-44c9-9a9b-0ccee74beb83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:07.716477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97796b98-8dc2-44c9-9a9b-0ccee74beb83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:07.718152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97796b98-8dc2-44c9-9a9b-0ccee74beb83 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-97796b98-8dc2-44c9-9a9b-0ccee74beb83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:42:07.730575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:07.732171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:07.733205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:42:07.733205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:42:07.733205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:42:07.733205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:42:07.733205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:42:07.733205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:42:07.733205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:07.733205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:07.733205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 605/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:07.739019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17930226-47bc-447b-9f52-908ae750b235 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:07.847195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17930226-47bc-447b-9f52-908ae750b235 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:07.870815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:07.882559 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e97481-b895-4aa6-a85a-c3502092b5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:07.883163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e97481-b895-4aa6-a85a-c3502092b5ae None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:07.883440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e97481-b895-4aa6-a85a-c3502092b5ae None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:07.883710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e97481-b895-4aa6-a85a-c3502092b5ae None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:07.884401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e97481-b895-4aa6-a85a-c3502092b5ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:07.884704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e97481-b895-4aa6-a85a-c3502092b5ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:07.886169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e97481-b895-4aa6-a85a-c3502092b5ae None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:07.898445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 613/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:42:07.908739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97796b98-8dc2-44c9-9a9b-0ccee74beb83 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:08.020837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97796b98-8dc2-44c9-9a9b-0ccee74beb83 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:08.060480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:08.067604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d86a8-d42e-40f6-8559-c7400fa8c1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:08.068348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d86a8-d42e-40f6-8559-c7400fa8c1b6 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:08.068719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d86a8-d42e-40f6-8559-c7400fa8c1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:08.069054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d86a8-d42e-40f6-8559-c7400fa8c1b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:08.069933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d86a8-d42e-40f6-8559-c7400fa8c1b6 None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=ebc01c87551044818e1ccbb46b8e7340) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:08.070332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d86a8-d42e-40f6-8559-c7400fa8c1b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:08.072004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d86a8-d42e-40f6-8559-c7400fa8c1b6 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:08.079155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 606/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:42:07 2022] GET /identity/v3/role_assignments?user.id=ebc01c87551044818e1ccbb46b8e7340&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:10.387670 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c0b26-77cc-4dbd-a37e-ac4cb5fcb54b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:10.388236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c0b26-77cc-4dbd-a37e-ac4cb5fcb54b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:10.388335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c0b26-77cc-4dbd-a37e-ac4cb5fcb54b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:10.390101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 614/1220] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:10.395178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159e1eb-3275-420f-9446-253dd5e35947 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:10.395645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159e1eb-3275-420f-9446-253dd5e35947 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:10.395828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159e1eb-3275-420f-9446-253dd5e35947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:10.417457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9159e1eb-3275-420f-9446-253dd5e35947 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:10.419558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9159e1eb-3275-420f-9446-253dd5e35947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:10.494198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 607/1221] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:42:10.787005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c0b26-77cc-4dbd-a37e-ac4cb5fcb54b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:10.893946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c0b26-77cc-4dbd-a37e-ac4cb5fcb54b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:10.928828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bf29c-a772-4dfd-9049-e4a06b3cd618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:10.929460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bf29c-a772-4dfd-9049-e4a06b3cd618 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:10.929776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bf29c-a772-4dfd-9049-e4a06b3cd618 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:10.930048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bf29c-a772-4dfd-9049-e4a06b3cd618 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:10.943023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bf29c-a772-4dfd-9049-e4a06b3cd618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-b27bf29c-a772-4dfd-9049-e4a06b3cd618 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:42:10.951217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:10.952298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:10.953140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 615/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:42:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:10.959306 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159e1eb-3275-420f-9446-253dd5e35947 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:11.068798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159e1eb-3275-420f-9446-253dd5e35947 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:11.096573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:11.108480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27d841-c649-4e7b-bae7-e7eaef01ca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:11.109094 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27d841-c649-4e7b-bae7-e7eaef01ca89 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:11.109655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27d841-c649-4e7b-bae7-e7eaef01ca89 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:11.109933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27d841-c649-4e7b-bae7-e7eaef01ca89 None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:11.110609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27d841-c649-4e7b-bae7-e7eaef01ca89 None admin] RBAC: Adding query filter params (name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:11.110915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27d841-c649-4e7b-bae7-e7eaef01ca89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:11.112330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27d841-c649-4e7b-bae7-e7eaef01ca89 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:11.120727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 608/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:42:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:11.126650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bf29c-a772-4dfd-9049-e4a06b3cd618 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:11.175222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:11.244053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bf29c-a772-4dfd-9049-e4a06b3cd618 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:11.286128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6db0df-be0e-4c3b-8160-fbf490631225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:11.286885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6db0df-be0e-4c3b-8160-fbf490631225 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:11.287175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6db0df-be0e-4c3b-8160-fbf490631225 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:11.287566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6db0df-be0e-4c3b-8160-fbf490631225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:11.306070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6db0df-be0e-4c3b-8160-fbf490631225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:11.307510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6db0df-be0e-4c3b-8160-fbf490631225 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-7a6db0df-be0e-4c3b-8160-fbf490631225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:11.323759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:11.324921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:11.330709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:11.330709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:11.330709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:11.330709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 616/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:42:11.330994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27d841-c649-4e7b-bae7-e7eaef01ca89 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:11.437864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27d841-c649-4e7b-bae7-e7eaef01ca89 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:11.473754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db1d59-d5b1-4f6d-82b4-9023f696eab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:11.474359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1d59-d5b1-4f6d-82b4-9023f696eab3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:11.474628 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1d59-d5b1-4f6d-82b4-9023f696eab3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:11.474902 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1d59-d5b1-4f6d-82b4-9023f696eab3 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:11.475663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db1d59-d5b1-4f6d-82b4-9023f696eab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:11.475971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db1d59-d5b1-4f6d-82b4-9023f696eab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:11.477525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db1d59-d5b1-4f6d-82b4-9023f696eab3 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:11.495451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 609/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:11.502694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6db0df-be0e-4c3b-8160-fbf490631225 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:11.627292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:11.635558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6db0df-be0e-4c3b-8160-fbf490631225 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:11.672482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2496c0-ffc9-4b20-b3cd-2eb1b340d757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:11.673138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2496c0-ffc9-4b20-b3cd-2eb1b340d757 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:11.673488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2496c0-ffc9-4b20-b3cd-2eb1b340d757 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:11.673814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2496c0-ffc9-4b20-b3cd-2eb1b340d757 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:11.685100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2496c0-ffc9-4b20-b3cd-2eb1b340d757 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:11.687690 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2496c0-ffc9-4b20-b3cd-2eb1b340d757 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-cb2496c0-ffc9-4b20-b3cd-2eb1b340d757 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:42:11.702494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:11.703917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:11.705044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 617/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:42:11.710003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db1d59-d5b1-4f6d-82b4-9023f696eab3 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:11.768509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:11.823607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db1d59-d5b1-4f6d-82b4-9023f696eab3 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:11.861828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b6a20-bad5-4ff2-ab8e-ff30f6f1c72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:11.862495 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b6a20-bad5-4ff2-ab8e-ff30f6f1c72d None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:11.862778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b6a20-bad5-4ff2-ab8e-ff30f6f1c72d None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:11.863184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b6a20-bad5-4ff2-ab8e-ff30f6f1c72d None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:11.863978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b6a20-bad5-4ff2-ab8e-ff30f6f1c72d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:11.864366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b6a20-bad5-4ff2-ab8e-ff30f6f1c72d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:11.865962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b6a20-bad5-4ff2-ab8e-ff30f6f1c72d None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:11.884191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 610/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 3 14:42:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:11.890461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2496c0-ffc9-4b20-b3cd-2eb1b340d757 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:11.995354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2496c0-ffc9-4b20-b3cd-2eb1b340d757 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:12.034265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795a636-a2ec-4438-b8aa-1f815ab08f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:12.034885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795a636-a2ec-4438-b8aa-1f815ab08f29 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:12.035162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795a636-a2ec-4438-b8aa-1f815ab08f29 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:12.035427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795a636-a2ec-4438-b8aa-1f815ab08f29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:12.046767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795a636-a2ec-4438-b8aa-1f815ab08f29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:12.048200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795a636-a2ec-4438-b8aa-1f815ab08f29 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-1795a636-a2ec-4438-b8aa-1f815ab08f29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:12.064732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:12.065985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:12.066886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:12.066886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:12.066886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:12.066886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 618/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:42:12.072592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b6a20-bad5-4ff2-ab8e-ff30f6f1c72d None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:12.179555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b6a20-bad5-4ff2-ab8e-ff30f6f1c72d None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:12.182117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:12.219878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73396b-6a83-4a0a-a734-5e023ed9d8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:12.220529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73396b-6a83-4a0a-a734-5e023ed9d8da None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:12.221020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73396b-6a83-4a0a-a734-5e023ed9d8da None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:12.221429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73396b-6a83-4a0a-a734-5e023ed9d8da None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:12.222264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73396b-6a83-4a0a-a734-5e023ed9d8da None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:12.222645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73396b-6a83-4a0a-a734-5e023ed9d8da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:12.224508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73396b-6a83-4a0a-a734-5e023ed9d8da None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:12.239558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 611/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:12.245377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795a636-a2ec-4438-b8aa-1f815ab08f29 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:12.255852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:12.360759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795a636-a2ec-4438-b8aa-1f815ab08f29 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:12.399618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae820f3-9e91-4e72-a13e-98052a9afd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:12.400288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae820f3-9e91-4e72-a13e-98052a9afd62 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:12.400566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae820f3-9e91-4e72-a13e-98052a9afd62 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:12.400834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae820f3-9e91-4e72-a13e-98052a9afd62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:12.407953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae820f3-9e91-4e72-a13e-98052a9afd62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:12.409388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae820f3-9e91-4e72-a13e-98052a9afd62 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-9ae820f3-9e91-4e72-a13e-98052a9afd62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:42:12.420257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:12.421298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:12.422250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:42:12.422250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:42:12.422250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:42:12.422250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:42:12.422250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:42:12.422250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:42:12.422250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:12.422250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:12.422250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 619/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:42:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:12.429712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73396b-6a83-4a0a-a734-5e023ed9d8da None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:12.540051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73396b-6a83-4a0a-a734-5e023ed9d8da None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:12.586758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e85b2-efea-4dbd-809a-b7ef871732d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:12.588072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e85b2-efea-4dbd-809a-b7ef871732d4 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:12.588560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e85b2-efea-4dbd-809a-b7ef871732d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:12.588962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e85b2-efea-4dbd-809a-b7ef871732d4 None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:12.590430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e85b2-efea-4dbd-809a-b7ef871732d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:12.591159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e85b2-efea-4dbd-809a-b7ef871732d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:12.594421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e85b2-efea-4dbd-809a-b7ef871732d4 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:12.611033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 612/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:42:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:12.617030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae820f3-9e91-4e72-a13e-98052a9afd62 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:12.690353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:12.731325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae820f3-9e91-4e72-a13e-98052a9afd62 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:12.771309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839e52d-d373-43ad-adfc-00ccb27f84e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:12.772028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839e52d-d373-43ad-adfc-00ccb27f84e2 None admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:12.772347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839e52d-d373-43ad-adfc-00ccb27f84e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:12.772680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839e52d-d373-43ad-adfc-00ccb27f84e2 None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/ebc01c87551044818e1ccbb46b8e7340/roles/f5822988729a45539e07519e59cf7bc4` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:12.789787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839e52d-d373-43ad-adfc-00ccb27f84e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=ebc01c87551044818e1ccbb46b8e7340, role_id=f5822988729a45539e07519e59cf7bc4)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:12.791272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839e52d-d373-43ad-adfc-00ccb27f84e2 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:12.804256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 620/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:42:12 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/ebc01c87551044818e1ccbb46b8e7340/roles/f5822988729a45539e07519e59cf7bc4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:42:14.927592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d2a74-d7bf-49db-80d4-e23475df1f7a None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:14.928226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d2a74-d7bf-49db-80d4-e23475df1f7a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:14.928226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d2a74-d7bf-49db-80d4-e23475df1f7a None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:14.929465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 613/1233] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:14.935203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e54e66-c0cd-441a-b987-e5ffca59fb91 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:14.935632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e54e66-c0cd-441a-b987-e5ffca59fb91 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:14.935830 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e54e66-c0cd-441a-b987-e5ffca59fb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:14.956587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-50e54e66-c0cd-441a-b987-e5ffca59fb91 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:14.957981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-50e54e66-c0cd-441a-b987-e5ffca59fb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:15.039562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 621/1234] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 14:42:15.338092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d2a74-d7bf-49db-80d4-e23475df1f7a None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:15.367562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:15.458336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d2a74-d7bf-49db-80d4-e23475df1f7a None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:15.498697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c09590-21ed-4a85-9463-0a135a68b706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:15.499330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09590-21ed-4a85-9463-0a135a68b706 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:15.499610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09590-21ed-4a85-9463-0a135a68b706 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:15.499878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09590-21ed-4a85-9463-0a135a68b706 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:15.510710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c09590-21ed-4a85-9463-0a135a68b706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-c5c09590-21ed-4a85-9463-0a135a68b706 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:42:15.516335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:15.517291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:15.518352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 614/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 3 14:42:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:42:15.523851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e54e66-c0cd-441a-b987-e5ffca59fb91 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:15.570722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:15.631227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e54e66-c0cd-441a-b987-e5ffca59fb91 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:15.672874 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130abb4-6d59-4991-be60-47df890e34b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:15.673571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130abb4-6d59-4991-be60-47df890e34b8 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:15.673886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130abb4-6d59-4991-be60-47df890e34b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:15.674188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130abb4-6d59-4991-be60-47df890e34b8 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:15.674950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130abb4-6d59-4991-be60-47df890e34b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:15.675288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130abb4-6d59-4991-be60-47df890e34b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:15.676795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130abb4-6d59-4991-be60-47df890e34b8 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:15.686422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 622/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 3 14:42:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:15.692772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c09590-21ed-4a85-9463-0a135a68b706 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:15.786702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:15.803990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c09590-21ed-4a85-9463-0a135a68b706 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:15.842556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0e672-8fa4-4cdd-aca9-938d6ffc61d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:15.843221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0e672-8fa4-4cdd-aca9-938d6ffc61d6 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:15.843571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0e672-8fa4-4cdd-aca9-938d6ffc61d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:15.843829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0e672-8fa4-4cdd-aca9-938d6ffc61d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:15.855344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0e672-8fa4-4cdd-aca9-938d6ffc61d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:15.857651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0e672-8fa4-4cdd-aca9-938d6ffc61d6 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-88f0e672-8fa4-4cdd-aca9-938d6ffc61d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:15.875652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:15.876687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:15.877604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:15.877604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:15.877604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:15.881800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 615/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:42:15.882867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130abb4-6d59-4991-be60-47df890e34b8 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:15.991097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130abb4-6d59-4991-be60-47df890e34b8 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:16.001419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:16.035807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10674-ba68-4124-a5c5-a87b9ba8d633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:16.036454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10674-ba68-4124-a5c5-a87b9ba8d633 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:16.036798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10674-ba68-4124-a5c5-a87b9ba8d633 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:16.037282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10674-ba68-4124-a5c5-a87b9ba8d633 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:16.038350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d10674-ba68-4124-a5c5-a87b9ba8d633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:16.038793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d10674-ba68-4124-a5c5-a87b9ba8d633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:16.043034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d10674-ba68-4124-a5c5-a87b9ba8d633 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:16.055744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 623/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:16.062356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0e672-8fa4-4cdd-aca9-938d6ffc61d6 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:16.176788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0e672-8fa4-4cdd-aca9-938d6ffc61d6 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:16.214026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee320e6e-e265-408d-afca-59998930a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:16.214716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320e6e-e265-408d-afca-59998930a113 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:16.215480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320e6e-e265-408d-afca-59998930a113 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:16.215759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320e6e-e265-408d-afca-59998930a113 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:16.219068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:16.228192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee320e6e-e265-408d-afca-59998930a113 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:16.229782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee320e6e-e265-408d-afca-59998930a113 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-ee320e6e-e265-408d-afca-59998930a113 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:42:16.241661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:16.242774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:16.244203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 616/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:16.249987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10674-ba68-4124-a5c5-a87b9ba8d633 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:16.366066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10674-ba68-4124-a5c5-a87b9ba8d633 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:16.409556 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44595c33-eeaa-447a-aacb-1760af66f330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:16.410222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44595c33-eeaa-447a-aacb-1760af66f330 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:16.410552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44595c33-eeaa-447a-aacb-1760af66f330 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:16.410878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44595c33-eeaa-447a-aacb-1760af66f330 None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:16.411713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44595c33-eeaa-447a-aacb-1760af66f330 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:16.412026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44595c33-eeaa-447a-aacb-1760af66f330 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:16.413771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44595c33-eeaa-447a-aacb-1760af66f330 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:16.435852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 624/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 3 14:42:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:16.442094 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee320e6e-e265-408d-afca-59998930a113 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:16.557785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee320e6e-e265-408d-afca-59998930a113 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:16.594763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3335e8-9916-49dc-b46e-166cc37cfb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:16.595417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3335e8-9916-49dc-b46e-166cc37cfb57 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:16.595841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3335e8-9916-49dc-b46e-166cc37cfb57 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:16.596113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3335e8-9916-49dc-b46e-166cc37cfb57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:16.663226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:16.671730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3335e8-9916-49dc-b46e-166cc37cfb57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:16.673370 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3335e8-9916-49dc-b46e-166cc37cfb57 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:16.785247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 617/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:42:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:42:16.791253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44595c33-eeaa-447a-aacb-1760af66f330 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:16.818623 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:16.900030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44595c33-eeaa-447a-aacb-1760af66f330 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:16.940954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0102b-8bd6-4168-aa3a-ffa3d35d0cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:16.941745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0102b-8bd6-4168-aa3a-ffa3d35d0cbe None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:16.942031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0102b-8bd6-4168-aa3a-ffa3d35d0cbe None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:16.942380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0102b-8bd6-4168-aa3a-ffa3d35d0cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:16.953434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0102b-8bd6-4168-aa3a-ffa3d35d0cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:16.954932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0102b-8bd6-4168-aa3a-ffa3d35d0cbe None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-34e0102b-8bd6-4168-aa3a-ffa3d35d0cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:16.971895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:16.978996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:16.980115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:16.980115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:16.980115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:16.980115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 625/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:16.980403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3335e8-9916-49dc-b46e-166cc37cfb57 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:17.098295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:17.104782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3335e8-9916-49dc-b46e-166cc37cfb57 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:17.144359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a639926-c06a-4921-bc12-3e5dfc425f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:17.145117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a639926-c06a-4921-bc12-3e5dfc425f43 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:17.145394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a639926-c06a-4921-bc12-3e5dfc425f43 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:17.145755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a639926-c06a-4921-bc12-3e5dfc425f43 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:17.146523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a639926-c06a-4921-bc12-3e5dfc425f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:17.146833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a639926-c06a-4921-bc12-3e5dfc425f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:17.148394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a639926-c06a-4921-bc12-3e5dfc425f43 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:17.161253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 618/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:17.167693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0102b-8bd6-4168-aa3a-ffa3d35d0cbe None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:17.254316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:17.280933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0102b-8bd6-4168-aa3a-ffa3d35d0cbe None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:17.318443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb5ebc-75a3-4504-8d37-5f8b40c14352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:17.319076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5ebc-75a3-4504-8d37-5f8b40c14352 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:17.319357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5ebc-75a3-4504-8d37-5f8b40c14352 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:17.319622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5ebc-75a3-4504-8d37-5f8b40c14352 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:17.327263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb5ebc-75a3-4504-8d37-5f8b40c14352 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:17.328704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb5ebc-75a3-4504-8d37-5f8b40c14352 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-66eb5ebc-75a3-4504-8d37-5f8b40c14352 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:42:17.341110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:17.343849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:17.345007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:42:17.345007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:42:17.345007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:42:17.345007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:42:17.345007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:42:17.345007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:42:17.345007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:17.345007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:17.345007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 626/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:42:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:17.348441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a639926-c06a-4921-bc12-3e5dfc425f43 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:17.464877 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a639926-c06a-4921-bc12-3e5dfc425f43 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:17.503586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d3c8f-90db-49e9-95fe-4c6684b3ee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:17.504246 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d3c8f-90db-49e9-95fe-4c6684b3ee4f None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:17.504557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d3c8f-90db-49e9-95fe-4c6684b3ee4f None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:17.504841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d3c8f-90db-49e9-95fe-4c6684b3ee4f None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:17.505801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d3c8f-90db-49e9-95fe-4c6684b3ee4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:17.506175 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d3c8f-90db-49e9-95fe-4c6684b3ee4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:17.507814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d3c8f-90db-49e9-95fe-4c6684b3ee4f None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:17.521649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 619/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:42:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:17.528561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb5ebc-75a3-4504-8d37-5f8b40c14352 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:17.635106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb5ebc-75a3-4504-8d37-5f8b40c14352 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:17.673376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879ffb6-5b56-4939-9e47-b87eba5070a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:17.674014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879ffb6-5b56-4939-9e47-b87eba5070a4 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:17.674292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879ffb6-5b56-4939-9e47-b87eba5070a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:17.674564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879ffb6-5b56-4939-9e47-b87eba5070a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:17.675287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879ffb6-5b56-4939-9e47-b87eba5070a4 None admin] RBAC: Adding query filter params (role.id=f5822988729a45539e07519e59cf7bc4, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=ebc01c87551044818e1ccbb46b8e7340) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:17.675597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879ffb6-5b56-4939-9e47-b87eba5070a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:17.677022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879ffb6-5b56-4939-9e47-b87eba5070a4 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:17.682959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 627/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:42:17 2022] GET /identity/v3/role_assignments?user.id=ebc01c87551044818e1ccbb46b8e7340&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=f5822988729a45539e07519e59cf7bc4 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:19.989715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04f534-f1a3-4594-9eec-3ddcd44304a1 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:19.990248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04f534-f1a3-4594-9eec-3ddcd44304a1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:19.990248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04f534-f1a3-4594-9eec-3ddcd44304a1 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:19.991577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 620/1247] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:19.997246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217f40e-0757-4cb5-8c2b-dff4dafd61db None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:19.997704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217f40e-0757-4cb5-8c2b-dff4dafd61db None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:19.997845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217f40e-0757-4cb5-8c2b-dff4dafd61db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:20.004094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:20.017458 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e217f40e-0757-4cb5-8c2b-dff4dafd61db None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:20.018801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e217f40e-0757-4cb5-8c2b-dff4dafd61db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:20.101780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 628/1248] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:42:20.384770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04f534-f1a3-4594-9eec-3ddcd44304a1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:20.405686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:20.500621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04f534-f1a3-4594-9eec-3ddcd44304a1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:20.539099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f1d58-b4c7-4f9f-844a-d30996cb0d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:20.539732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f1d58-b4c7-4f9f-844a-d30996cb0d83 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:20.540008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f1d58-b4c7-4f9f-844a-d30996cb0d83 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:20.540280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f1d58-b4c7-4f9f-844a-d30996cb0d83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:20.553684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f1d58-b4c7-4f9f-844a-d30996cb0d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-dc0f1d58-b4c7-4f9f-844a-d30996cb0d83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:42:20.559459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:20.560594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:20.561660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 621/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:42:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:20.567447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217f40e-0757-4cb5-8c2b-dff4dafd61db None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:20.673809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217f40e-0757-4cb5-8c2b-dff4dafd61db None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:20.708299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:20.714368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf93187-04e4-4a04-b2ec-98426364f38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:20.714996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf93187-04e4-4a04-b2ec-98426364f38f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:20.715280 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf93187-04e4-4a04-b2ec-98426364f38f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:20.715550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf93187-04e4-4a04-b2ec-98426364f38f None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:20.716218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf93187-04e4-4a04-b2ec-98426364f38f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:20.716548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf93187-04e4-4a04-b2ec-98426364f38f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:20.717939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf93187-04e4-4a04-b2ec-98426364f38f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:20.726345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 629/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:42:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:20.733113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f1d58-b4c7-4f9f-844a-d30996cb0d83 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:20.819864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:20.844750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f1d58-b4c7-4f9f-844a-d30996cb0d83 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:20.880959 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c19d1-ae25-404e-8d23-273c5cb975bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:20.881635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c19d1-ae25-404e-8d23-273c5cb975bb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:20.881922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c19d1-ae25-404e-8d23-273c5cb975bb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:20.882220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c19d1-ae25-404e-8d23-273c5cb975bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:20.893322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c19d1-ae25-404e-8d23-273c5cb975bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:20.894827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c19d1-ae25-404e-8d23-273c5cb975bb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-953c19d1-ae25-404e-8d23-273c5cb975bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:20.910774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:20.912012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:20.913050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:20.913050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:20.913050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:20.913050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 622/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:20.918588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf93187-04e4-4a04-b2ec-98426364f38f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:21.023338 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf93187-04e4-4a04-b2ec-98426364f38f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:21.066946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a66c0-d4ad-4d83-a5e9-2e2b6783c519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:21.067541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a66c0-d4ad-4d83-a5e9-2e2b6783c519 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:21.067850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a66c0-d4ad-4d83-a5e9-2e2b6783c519 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:21.068188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a66c0-d4ad-4d83-a5e9-2e2b6783c519 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:21.068808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a66c0-d4ad-4d83-a5e9-2e2b6783c519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:21.069113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a66c0-d4ad-4d83-a5e9-2e2b6783c519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:21.070907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a66c0-d4ad-4d83-a5e9-2e2b6783c519 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:21.085744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 630/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:21.092206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c19d1-ae25-404e-8d23-273c5cb975bb None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:21.193780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c19d1-ae25-404e-8d23-273c5cb975bb None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:21.220568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:21.229241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49674113-cc61-40b6-9da6-2468f0bb75cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:21.229892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49674113-cc61-40b6-9da6-2468f0bb75cd None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:21.230169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49674113-cc61-40b6-9da6-2468f0bb75cd None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:21.230499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49674113-cc61-40b6-9da6-2468f0bb75cd None admin] PATH_INFO: `/v3/users/placement` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:21.238193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49674113-cc61-40b6-9da6-2468f0bb75cd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:21.239612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49674113-cc61-40b6-9da6-2468f0bb75cd None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-49674113-cc61-40b6-9da6-2468f0bb75cd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:42:21.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:21.251662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:21.252754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 623/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:21.257616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a66c0-d4ad-4d83-a5e9-2e2b6783c519 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:21.315694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:21.361796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a66c0-d4ad-4d83-a5e9-2e2b6783c519 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:21.399656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6cef9-7ee5-482c-b1e8-7651bbb9ae9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:21.400263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6cef9-7ee5-482c-b1e8-7651bbb9ae9a None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:21.400546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6cef9-7ee5-482c-b1e8-7651bbb9ae9a None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:21.400823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6cef9-7ee5-482c-b1e8-7651bbb9ae9a None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:21.401575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6cef9-7ee5-482c-b1e8-7651bbb9ae9a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:21.401882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6cef9-7ee5-482c-b1e8-7651bbb9ae9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:21.403909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6cef9-7ee5-482c-b1e8-7651bbb9ae9a None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:21.422489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 631/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 3 14:42:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:21.428285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49674113-cc61-40b6-9da6-2468f0bb75cd None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:21.523894 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49674113-cc61-40b6-9da6-2468f0bb75cd None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:21.555526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae362106-2de8-4c74-9a81-4143e776d500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:21.556107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae362106-2de8-4c74-9a81-4143e776d500 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:21.556384 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae362106-2de8-4c74-9a81-4143e776d500 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:21.556651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae362106-2de8-4c74-9a81-4143e776d500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:21.608321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:21.620698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae362106-2de8-4c74-9a81-4143e776d500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:21.622626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae362106-2de8-4c74-9a81-4143e776d500 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:21.722970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 624/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:42:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:42:21.728914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6cef9-7ee5-482c-b1e8-7651bbb9ae9a None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:21.823738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6cef9-7ee5-482c-b1e8-7651bbb9ae9a None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:21.836689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:21.857279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a038e9-799b-4faf-91b0-0550f6b14add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:21.857922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a038e9-799b-4faf-91b0-0550f6b14add None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:21.858201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a038e9-799b-4faf-91b0-0550f6b14add None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:21.858468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a038e9-799b-4faf-91b0-0550f6b14add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:21.868346 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a038e9-799b-4faf-91b0-0550f6b14add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:21.869815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a038e9-799b-4faf-91b0-0550f6b14add None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-a0a038e9-799b-4faf-91b0-0550f6b14add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:21.883779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:21.884857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:21.886010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:21.886010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:21.886010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:21.886010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 632/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:21.891073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae362106-2de8-4c74-9a81-4143e776d500 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:21.974016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:21.986259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae362106-2de8-4c74-9a81-4143e776d500 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:22.020771 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a9f1f-aa05-4f06-b2cf-69234ceb80b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:22.021475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a9f1f-aa05-4f06-b2cf-69234ceb80b8 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:22.021827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a9f1f-aa05-4f06-b2cf-69234ceb80b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:22.022151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a9f1f-aa05-4f06-b2cf-69234ceb80b8 None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:22.022869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a9f1f-aa05-4f06-b2cf-69234ceb80b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:22.023259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a9f1f-aa05-4f06-b2cf-69234ceb80b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:22.024843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a9f1f-aa05-4f06-b2cf-69234ceb80b8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:22.038561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 625/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:22.044944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a038e9-799b-4faf-91b0-0550f6b14add None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:22.146926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a038e9-799b-4faf-91b0-0550f6b14add None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:22.178714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0833796-eafa-4129-a7ee-197871c1145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:22.179324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0833796-eafa-4129-a7ee-197871c1145c None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:22.179598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0833796-eafa-4129-a7ee-197871c1145c None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:22.179865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0833796-eafa-4129-a7ee-197871c1145c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:22.186685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0833796-eafa-4129-a7ee-197871c1145c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:22.188115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0833796-eafa-4129-a7ee-197871c1145c None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-c0833796-eafa-4129-a7ee-197871c1145c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:42:22.198824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:22.200367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:22.201263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:42:22.201263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:42:22.201263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:42:22.201263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:42:22.201263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:42:22.201263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:42:22.201263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:22.201263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:22.201263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 633/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:42:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:22.206350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a9f1f-aa05-4f06-b2cf-69234ceb80b8 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:22.301708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a9f1f-aa05-4f06-b2cf-69234ceb80b8 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:22.342927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d99cc-517f-4544-974b-d63660fdc8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:22.343526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d99cc-517f-4544-974b-d63660fdc8c0 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:22.343801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d99cc-517f-4544-974b-d63660fdc8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:22.344069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d99cc-517f-4544-974b-d63660fdc8c0 None admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:22.344762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d99cc-517f-4544-974b-d63660fdc8c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:22.345063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d99cc-517f-4544-974b-d63660fdc8c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:22.346498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d99cc-517f-4544-974b-d63660fdc8c0 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:22.357796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 626/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:22.364066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0833796-eafa-4129-a7ee-197871c1145c None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:22.389479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:22.460527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0833796-eafa-4129-a7ee-197871c1145c None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:22.492158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5869f-4d26-4c33-9d4f-66fcf1fa5d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:22.492736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5869f-4d26-4c33-9d4f-66fcf1fa5d65 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:22.493008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5869f-4d26-4c33-9d4f-66fcf1fa5d65 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:22.493271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5869f-4d26-4c33-9d4f-66fcf1fa5d65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:22.494007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5869f-4d26-4c33-9d4f-66fcf1fa5d65 None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=ebc01c87551044818e1ccbb46b8e7340) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:22.494309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5869f-4d26-4c33-9d4f-66fcf1fa5d65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:22.495729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5869f-4d26-4c33-9d4f-66fcf1fa5d65 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:22.501402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 634/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:42:22 2022] GET /identity/v3/role_assignments?user.id=ebc01c87551044818e1ccbb46b8e7340&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:24.624096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ddb05-0d65-4784-bf4e-53c8692dfddd None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:24.624606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ddb05-0d65-4784-bf4e-53c8692dfddd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:24.624965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ddb05-0d65-4784-bf4e-53c8692dfddd None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:24.627391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 627/1261] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:24.632619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774106ca-ddf9-45ee-a850-7d70a3b63413 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:24.633039 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774106ca-ddf9-45ee-a850-7d70a3b63413 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:24.633271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774106ca-ddf9-45ee-a850-7d70a3b63413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:24.651498 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-774106ca-ddf9-45ee-a850-7d70a3b63413 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:24.653163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-774106ca-ddf9-45ee-a850-7d70a3b63413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:24.718049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 635/1262] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:42:24.984337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ddb05-0d65-4784-bf4e-53c8692dfddd None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:24.995065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:25.098643 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ddb05-0d65-4784-bf4e-53c8692dfddd None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:25.133278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a1de1-3dc1-4266-b998-d46e7bb8a2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:25.133921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1de1-3dc1-4266-b998-d46e7bb8a2b2 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:25.134197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1de1-3dc1-4266-b998-d46e7bb8a2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:25.134463 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1de1-3dc1-4266-b998-d46e7bb8a2b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:25.144559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a1de1-3dc1-4266-b998-d46e7bb8a2b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-980a1de1-3dc1-4266-b998-d46e7bb8a2b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:42:25.149225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:25.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:25.151592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 628/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:42:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:25.157296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774106ca-ddf9-45ee-a850-7d70a3b63413 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:25.175866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:25.258557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774106ca-ddf9-45ee-a850-7d70a3b63413 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:25.292431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f2aec-272e-4477-a2c5-5408a141cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:25.293072 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f2aec-272e-4477-a2c5-5408a141cf38 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:25.293351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f2aec-272e-4477-a2c5-5408a141cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:25.293654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f2aec-272e-4477-a2c5-5408a141cf38 None admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:25.294446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f2aec-272e-4477-a2c5-5408a141cf38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:25.294617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f2aec-272e-4477-a2c5-5408a141cf38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:25.295976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f2aec-272e-4477-a2c5-5408a141cf38 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:25.304150 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 636/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:42:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:25.310423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a1de1-3dc1-4266-b998-d46e7bb8a2b2 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:25.378776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:25.411988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a1de1-3dc1-4266-b998-d46e7bb8a2b2 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:25.445262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c785a0d-087c-46f2-994d-cb14e03d9b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:25.446428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c785a0d-087c-46f2-994d-cb14e03d9b84 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:25.446716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c785a0d-087c-46f2-994d-cb14e03d9b84 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:25.446993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c785a0d-087c-46f2-994d-cb14e03d9b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:25.457266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c785a0d-087c-46f2-994d-cb14e03d9b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:25.458743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c785a0d-087c-46f2-994d-cb14e03d9b84 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-0c785a0d-087c-46f2-994d-cb14e03d9b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:25.472867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:25.473955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:25.475052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:25.475052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:25.475052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:25.475052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 629/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:25.480043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f2aec-272e-4477-a2c5-5408a141cf38 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:25.559586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:25.578364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f2aec-272e-4477-a2c5-5408a141cf38 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:25.611064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9684341-5229-4441-81fc-ec624eb7f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:25.611659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9684341-5229-4441-81fc-ec624eb7f627 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:25.611941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9684341-5229-4441-81fc-ec624eb7f627 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:25.612214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9684341-5229-4441-81fc-ec624eb7f627 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:25.612849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9684341-5229-4441-81fc-ec624eb7f627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:25.613152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9684341-5229-4441-81fc-ec624eb7f627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:25.614548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9684341-5229-4441-81fc-ec624eb7f627 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:25.626416 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 637/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:25.632002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c785a0d-087c-46f2-994d-cb14e03d9b84 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:25.726990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c785a0d-087c-46f2-994d-cb14e03d9b84 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:25.746871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:25.761041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58b1dc-e433-4c32-9cf1-2a27b27b786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:25.761687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58b1dc-e433-4c32-9cf1-2a27b27b786e None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:25.761970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58b1dc-e433-4c32-9cf1-2a27b27b786e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:25.762239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58b1dc-e433-4c32-9cf1-2a27b27b786e None admin] PATH_INFO: `/v3/users/placement` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:25.769439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58b1dc-e433-4c32-9cf1-2a27b27b786e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:25.770915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58b1dc-e433-4c32-9cf1-2a27b27b786e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-9e58b1dc-e433-4c32-9cf1-2a27b27b786e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:42:25.781598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:25.782833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:25.783908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 630/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:25.788363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9684341-5229-4441-81fc-ec624eb7f627 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:25.885218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9684341-5229-4441-81fc-ec624eb7f627 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:25.918285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c387722-2c3f-40c7-93b2-ac31b9cb7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:25.918877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c387722-2c3f-40c7-93b2-ac31b9cb7c8f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:25.919155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c387722-2c3f-40c7-93b2-ac31b9cb7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:25.919429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c387722-2c3f-40c7-93b2-ac31b9cb7c8f None admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:25.920143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c387722-2c3f-40c7-93b2-ac31b9cb7c8f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:25.920447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c387722-2c3f-40c7-93b2-ac31b9cb7c8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:25.921891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c387722-2c3f-40c7-93b2-ac31b9cb7c8f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:25.930150 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:25.938575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 638/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 3 14:42:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:25.944337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58b1dc-e433-4c32-9cf1-2a27b27b786e None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:26.047078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58b1dc-e433-4c32-9cf1-2a27b27b786e None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:26.078783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1a74b-b9db-48e7-82e8-713ad0d50fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:26.079396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a74b-b9db-48e7-82e8-713ad0d50fdb None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:26.079673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a74b-b9db-48e7-82e8-713ad0d50fdb None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:26.079941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a74b-b9db-48e7-82e8-713ad0d50fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:26.090432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1a74b-b9db-48e7-82e8-713ad0d50fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:26.091853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1a74b-b9db-48e7-82e8-713ad0d50fdb None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-56b1a74b-b9db-48e7-82e8-713ad0d50fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:26.105801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:26.106712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:26.107663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:26.107663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:26.107663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:26.107663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 631/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:26.113097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c387722-2c3f-40c7-93b2-ac31b9cb7c8f None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:26.208036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c387722-2c3f-40c7-93b2-ac31b9cb7c8f None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:26.241572 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd3d3c-e989-4024-a34c-c0bf97e02e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:26.242165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd3d3c-e989-4024-a34c-c0bf97e02e1d None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:26.242445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd3d3c-e989-4024-a34c-c0bf97e02e1d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:26.242718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd3d3c-e989-4024-a34c-c0bf97e02e1d None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:26.243353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd3d3c-e989-4024-a34c-c0bf97e02e1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:26.243658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd3d3c-e989-4024-a34c-c0bf97e02e1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:26.245017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd3d3c-e989-4024-a34c-c0bf97e02e1d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:26.255942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 639/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:26.261351 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1a74b-b9db-48e7-82e8-713ad0d50fdb None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:26.291959 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:26.359028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1a74b-b9db-48e7-82e8-713ad0d50fdb None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:26.390909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc3684-ac35-466c-bfa8-c87177c6dda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:26.391520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc3684-ac35-466c-bfa8-c87177c6dda7 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:26.391800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc3684-ac35-466c-bfa8-c87177c6dda7 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:26.392071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc3684-ac35-466c-bfa8-c87177c6dda7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:26.398360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc3684-ac35-466c-bfa8-c87177c6dda7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:26.399797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc3684-ac35-466c-bfa8-c87177c6dda7 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-e3fc3684-ac35-466c-bfa8-c87177c6dda7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:42:26.411151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:26.412220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:26.413245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:42:26.413245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:42:26.413245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:42:26.413245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:42:26.413245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:42:26.413245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:42:26.413245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:26.413245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:26.413245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 632/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:26.418041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd3d3c-e989-4024-a34c-c0bf97e02e1d None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:26.474634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:26.514751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd3d3c-e989-4024-a34c-c0bf97e02e1d None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:26.546829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f82bc-bda0-4f4f-ab97-72a29cc12527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:26.547418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f82bc-bda0-4f4f-ab97-72a29cc12527 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:26.547692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f82bc-bda0-4f4f-ab97-72a29cc12527 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:26.547958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f82bc-bda0-4f4f-ab97-72a29cc12527 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:26.548632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f82bc-bda0-4f4f-ab97-72a29cc12527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:26.548933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f82bc-bda0-4f4f-ab97-72a29cc12527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:26.550379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f82bc-bda0-4f4f-ab97-72a29cc12527 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:26.561902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 640/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:26.567802 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc3684-ac35-466c-bfa8-c87177c6dda7 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:26.655569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:26.664065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc3684-ac35-466c-bfa8-c87177c6dda7 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:26.695535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7ae29-1f16-4759-adf5-69cd79bfb9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:26.696167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7ae29-1f16-4759-adf5-69cd79bfb9c8 None admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:26.696443 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7ae29-1f16-4759-adf5-69cd79bfb9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:26.696710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7ae29-1f16-4759-adf5-69cd79bfb9c8 None admin] PATH_INFO: `/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/ebc01c87551044818e1ccbb46b8e7340/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:26.698571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7ae29-1f16-4759-adf5-69cd79bfb9c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a912636f54fc41d0b7fa1a150154b06a, user_id=ebc01c87551044818e1ccbb46b8e7340, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:26.700061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7ae29-1f16-4759-adf5-69cd79bfb9c8 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:26.711904 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 633/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 3 14:42:26 2022] PUT /identity/v3/projects/a912636f54fc41d0b7fa1a150154b06a/users/ebc01c87551044818e1ccbb46b8e7340/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:42:28.797917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81165f-cde8-4634-890b-90860494cb4a None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:28.798436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81165f-cde8-4634-890b-90860494cb4a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:28.798436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81165f-cde8-4634-890b-90860494cb4a None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:28.799784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 641/1274] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:28.804826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f4e44-9142-4a6b-9a65-545b04474dee None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:28.805189 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f4e44-9142-4a6b-9a65-545b04474dee None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:28.805374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f4e44-9142-4a6b-9a65-545b04474dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:28.823592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ee9f4e44-9142-4a6b-9a65-545b04474dee None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:28.824772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ee9f4e44-9142-4a6b-9a65-545b04474dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:28.888202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 634/1275] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:42:29.164820 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81165f-cde8-4634-890b-90860494cb4a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:29.260796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81165f-cde8-4634-890b-90860494cb4a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:29.278592 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:29.295764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef85a7e-da1d-4d6d-94c1-682e2ade2055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:29.296380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef85a7e-da1d-4d6d-94c1-682e2ade2055 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:29.296653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef85a7e-da1d-4d6d-94c1-682e2ade2055 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:29.296918 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef85a7e-da1d-4d6d-94c1-682e2ade2055 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:29.306578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef85a7e-da1d-4d6d-94c1-682e2ade2055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-fef85a7e-da1d-4d6d-94c1-682e2ade2055 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 14:42:29.311277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:29.312311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:29.313441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 642/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 3 14:42:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:29.318251 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f4e44-9142-4a6b-9a65-545b04474dee None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:29.395716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:29.414912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f4e44-9142-4a6b-9a65-545b04474dee None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:29.446624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d92704-8a9e-4fcf-9e94-f24985ffdfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:29.447220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d92704-8a9e-4fcf-9e94-f24985ffdfba None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:29.447498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d92704-8a9e-4fcf-9e94-f24985ffdfba None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:29.447769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d92704-8a9e-4fcf-9e94-f24985ffdfba None admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:29.448430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d92704-8a9e-4fcf-9e94-f24985ffdfba None admin] RBAC: Adding query filter params (name=admin) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:29.449257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d92704-8a9e-4fcf-9e94-f24985ffdfba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:29.450643 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d92704-8a9e-4fcf-9e94-f24985ffdfba None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:29.458330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 635/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 3 14:42:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:29.464126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef85a7e-da1d-4d6d-94c1-682e2ade2055 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:29.559115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef85a7e-da1d-4d6d-94c1-682e2ade2055 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:29.592020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd0878-7485-4cae-98a1-d58d1eda9ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:29.592617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd0878-7485-4cae-98a1-d58d1eda9ea6 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:29.592889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd0878-7485-4cae-98a1-d58d1eda9ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:29.593154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd0878-7485-4cae-98a1-d58d1eda9ea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:29.602745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd0878-7485-4cae-98a1-d58d1eda9ea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:29.604165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd0878-7485-4cae-98a1-d58d1eda9ea6 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-49dd0878-7485-4cae-98a1-d58d1eda9ea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:29.618329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:29.619440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:29.620509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:29.620509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:29.620509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:29.620509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 643/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:29.625027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d92704-8a9e-4fcf-9e94-f24985ffdfba None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:29.720884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d92704-8a9e-4fcf-9e94-f24985ffdfba None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:29.753621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808452d1-5eec-4599-8430-2045356378db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:29.754216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808452d1-5eec-4599-8430-2045356378db None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:29.754494 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808452d1-5eec-4599-8430-2045356378db None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:29.754764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808452d1-5eec-4599-8430-2045356378db None admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:29.755398 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808452d1-5eec-4599-8430-2045356378db None admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:29.755700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808452d1-5eec-4599-8430-2045356378db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:29.757055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808452d1-5eec-4599-8430-2045356378db None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:29.768963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 636/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:29.774513 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd0878-7485-4cae-98a1-d58d1eda9ea6 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:29.804089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:29.872167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd0878-7485-4cae-98a1-d58d1eda9ea6 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:29.904667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca9e29-63f4-42b5-85fc-9b64f75e789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:29.905281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9e29-63f4-42b5-85fc-9b64f75e789e None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:29.905590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9e29-63f4-42b5-85fc-9b64f75e789e None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:29.905863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9e29-63f4-42b5-85fc-9b64f75e789e None admin] PATH_INFO: `/v3/users/placement` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:29.912880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca9e29-63f4-42b5-85fc-9b64f75e789e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:29.914369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca9e29-63f4-42b5-85fc-9b64f75e789e None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-e6ca9e29-63f4-42b5-85fc-9b64f75e789e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 14:42:29.925193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:29.926278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:29.927627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 644/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 14:42:29.932024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808452d1-5eec-4599-8430-2045356378db None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:29.936252 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:30.029853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808452d1-5eec-4599-8430-2045356378db None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:30.063556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f42218-7947-4be3-a795-a957326d63bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:30.064153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f42218-7947-4be3-a795-a957326d63bc None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:30.064431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f42218-7947-4be3-a795-a957326d63bc None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:30.064728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f42218-7947-4be3-a795-a957326d63bc None admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:30.065454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f42218-7947-4be3-a795-a957326d63bc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:30.065790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f42218-7947-4be3-a795-a957326d63bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:30.067207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f42218-7947-4be3-a795-a957326d63bc None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:30.083348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 637/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 3 14:42:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:30.089208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca9e29-63f4-42b5-85fc-9b64f75e789e None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:30.176607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:30.186468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca9e29-63f4-42b5-85fc-9b64f75e789e None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:30.217719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8be855-2c49-4da4-9d46-3954ea48c6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:30.218302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8be855-2c49-4da4-9d46-3954ea48c6ff None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:30.218586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8be855-2c49-4da4-9d46-3954ea48c6ff None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:30.218856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8be855-2c49-4da4-9d46-3954ea48c6ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:30.282459 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8be855-2c49-4da4-9d46-3954ea48c6ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:30.283836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8be855-2c49-4da4-9d46-3954ea48c6ff None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:30.368453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:30.379221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 645/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 3 14:42:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 14:42:30.385090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f42218-7947-4be3-a795-a957326d63bc None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:30.444786 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:30.487368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f42218-7947-4be3-a795-a957326d63bc None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:30.519798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db311615-3406-40fd-b301-306d5a89816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:30.520408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db311615-3406-40fd-b301-306d5a89816a None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:30.520680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db311615-3406-40fd-b301-306d5a89816a None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:30.520944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db311615-3406-40fd-b301-306d5a89816a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:30.531015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db311615-3406-40fd-b301-306d5a89816a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:30.532431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db311615-3406-40fd-b301-306d5a89816a None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-db311615-3406-40fd-b301-306d5a89816a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 14:42:30.547325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:30.548444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 14:42:30.549383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 14:42:30.549383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 14:42:30.549383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:30.549383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 638/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:30.554081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8be855-2c49-4da4-9d46-3954ea48c6ff None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:30.648671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8be855-2c49-4da4-9d46-3954ea48c6ff None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:30.681675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeed4e-d056-4ff2-b286-4ed958ae9f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:30.682268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeed4e-d056-4ff2-b286-4ed958ae9f39 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:30.682549 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeed4e-d056-4ff2-b286-4ed958ae9f39 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:30.682822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeed4e-d056-4ff2-b286-4ed958ae9f39 None admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:30.683462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeed4e-d056-4ff2-b286-4ed958ae9f39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:30.683761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeed4e-d056-4ff2-b286-4ed958ae9f39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:30.685115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeed4e-d056-4ff2-b286-4ed958ae9f39 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:30.695832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 646/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 3 14:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:30.701244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db311615-3406-40fd-b301-306d5a89816a None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:30.799099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db311615-3406-40fd-b301-306d5a89816a None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:30.812316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:30.831886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdcbcd-60cc-4c29-896d-a7e2f06a06f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:30.832504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdcbcd-60cc-4c29-896d-a7e2f06a06f9 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:30.832779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdcbcd-60cc-4c29-896d-a7e2f06a06f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:30.833048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdcbcd-60cc-4c29-896d-a7e2f06a06f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:30.839932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cdcbcd-60cc-4c29-896d-a7e2f06a06f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:30.841349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cdcbcd-60cc-4c29-896d-a7e2f06a06f9 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application [None req-d2cdcbcd-60cc-4c29-896d-a7e2f06a06f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 14:42:30.851994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:30.853006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:30.859050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 14:42:30.859050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 14:42:30.859050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 14:42:30.859050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 14:42:30.859050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 14:42:30.859050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 14:42:30.859050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 14:42:30.859050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: ERROR keystone.server.flask.application Oct 03 14:42:30.859050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 639/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 3 14:42:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:30.859541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeed4e-d056-4ff2-b286-4ed958ae9f39 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:30.907098 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:30.955391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeed4e-d056-4ff2-b286-4ed958ae9f39 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:30.987892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b814092-80e5-49f2-8a01-69850ed8dd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:30.988479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b814092-80e5-49f2-8a01-69850ed8dd28 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:30.988755 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b814092-80e5-49f2-8a01-69850ed8dd28 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:30.989025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b814092-80e5-49f2-8a01-69850ed8dd28 None admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:30.989729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b814092-80e5-49f2-8a01-69850ed8dd28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:30.990033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b814092-80e5-49f2-8a01-69850ed8dd28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:30.991439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b814092-80e5-49f2-8a01-69850ed8dd28 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:31.002866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 647/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 3 14:42:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:31.008835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdcbcd-60cc-4c29-896d-a7e2f06a06f9 None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:31.106041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdcbcd-60cc-4c29-896d-a7e2f06a06f9 None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:31.137978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5079619-1884-4901-9b45-33cef136f9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:31.138561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5079619-1884-4901-9b45-33cef136f9d3 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:31.138865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5079619-1884-4901-9b45-33cef136f9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:31.139136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5079619-1884-4901-9b45-33cef136f9d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:31.139853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5079619-1884-4901-9b45-33cef136f9d3 None admin] RBAC: Adding query filter params (role.id=ea657fbc11e2449b943418b35bf4a4ff, scope.project.id=a912636f54fc41d0b7fa1a150154b06a, user.id=ebc01c87551044818e1ccbb46b8e7340) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:31.140162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5079619-1884-4901-9b45-33cef136f9d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:31.141602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5079619-1884-4901-9b45-33cef136f9d3 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:31.147048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 640/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 3 14:42:31 2022] GET /identity/v3/role_assignments?user.id=ebc01c87551044818e1ccbb46b8e7340&scope.project.id=a912636f54fc41d0b7fa1a150154b06a&role.id=ea657fbc11e2449b943418b35bf4a4ff => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:33.288996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b654e-07ff-4c9c-b4da-38c8f9d4d9f8 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:33.289581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b654e-07ff-4c9c-b4da-38c8f9d4d9f8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:33.289672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b654e-07ff-4c9c-b4da-38c8f9d4d9f8 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:33.291018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 648/1288] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:33.296280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f13fe-112e-42fa-94fe-34f4851e8844 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:33.296657 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f13fe-112e-42fa-94fe-34f4851e8844 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:33.296826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f13fe-112e-42fa-94fe-34f4851e8844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:33.314683 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d77f13fe-112e-42fa-94fe-34f4851e8844 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:33.315873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d77f13fe-112e-42fa-94fe-34f4851e8844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:33.344844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:33.380430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 641/1289] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:42:33.652579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b654e-07ff-4c9c-b4da-38c8f9d4d9f8 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:33.750987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b654e-07ff-4c9c-b4da-38c8f9d4d9f8 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:33.761881 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:33.787541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fbd69-6837-45e3-9e09-bf333c3a81af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:33.788220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fbd69-6837-45e3-9e09-bf333c3a81af None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:33.788546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fbd69-6837-45e3-9e09-bf333c3a81af None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:33.788823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fbd69-6837-45e3-9e09-bf333c3a81af None admin] PATH_INFO: `/v3/services/placement` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:33.789482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fbd69-6837-45e3-9e09-bf333c3a81af None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:33.791057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fbd69-6837-45e3-9e09-bf333c3a81af None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-a89fbd69-6837-45e3-9e09-bf333c3a81af None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:33.800533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:33.801720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:33.802662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 649/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 3 14:42:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:33.807595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f13fe-112e-42fa-94fe-34f4851e8844 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:33.900591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f13fe-112e-42fa-94fe-34f4851e8844 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:33.932430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75a590-74dc-41ce-9b7d-9a0a9426e8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:33.932984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75a590-74dc-41ce-9b7d-9a0a9426e8f4 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:33.933259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75a590-74dc-41ce-9b7d-9a0a9426e8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:33.933545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75a590-74dc-41ce-9b7d-9a0a9426e8f4 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:33.934168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75a590-74dc-41ce-9b7d-9a0a9426e8f4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:33.934466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75a590-74dc-41ce-9b7d-9a0a9426e8f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:33.935804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75a590-74dc-41ce-9b7d-9a0a9426e8f4 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:33.940349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 642/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Mon Oct 3 14:42:33 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:33.946017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fbd69-6837-45e3-9e09-bf333c3a81af None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:34.048732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fbd69-6837-45e3-9e09-bf333c3a81af None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:34.088192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a71a1-7609-4ac6-aed8-150f0a1f5c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:34.088862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a71a1-7609-4ac6-aed8-150f0a1f5c39 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:34.089814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a71a1-7609-4ac6-aed8-150f0a1f5c39 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:34.090133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a71a1-7609-4ac6-aed8-150f0a1f5c39 None admin] PATH_INFO: `/v3/services` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:34.090940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a71a1-7609-4ac6-aed8-150f0a1f5c39 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:34.091438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a71a1-7609-4ac6-aed8-150f0a1f5c39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:34.092899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a71a1-7609-4ac6-aed8-150f0a1f5c39 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:34.097374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 650/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Mon Oct 3 14:42:33 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:36.213595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4d35a-bb85-4f5c-8d0e-f3f8f733bb8a None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:36.214200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4d35a-bb85-4f5c-8d0e-f3f8f733bb8a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:36.214200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4d35a-bb85-4f5c-8d0e-f3f8f733bb8a None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:36.215537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 643/1293] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:36.220626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db285016-e281-4958-ba57-8bf22d1097a5 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:36.221008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db285016-e281-4958-ba57-8bf22d1097a5 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:36.221180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db285016-e281-4958-ba57-8bf22d1097a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:36.239678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-db285016-e281-4958-ba57-8bf22d1097a5 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:36.240855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-db285016-e281-4958-ba57-8bf22d1097a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:36.269820 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:36.306533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 651/1294] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:42:36.574462 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4d35a-bb85-4f5c-8d0e-f3f8f733bb8a None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:36.605408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:36.674607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4d35a-bb85-4f5c-8d0e-f3f8f733bb8a None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:36.706531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58949993-2828-40c2-b4e4-5c0049b99d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:36.707125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58949993-2828-40c2-b4e4-5c0049b99d1e None admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:36.707405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58949993-2828-40c2-b4e4-5c0049b99d1e None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:36.707675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58949993-2828-40c2-b4e4-5c0049b99d1e None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:36.708478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58949993-2828-40c2-b4e4-5c0049b99d1e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:36.709874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58949993-2828-40c2-b4e4-5c0049b99d1e None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:36.724019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 644/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 3 14:42:36 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:42:38.839487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eb849-7265-4c9a-b687-5848878f242b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:38.840044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eb849-7265-4c9a-b687-5848878f242b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:38.840044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eb849-7265-4c9a-b687-5848878f242b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:38.841346 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 652/1296] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:38.846611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38595c0-4177-4800-8139-bf0d2fb971b9 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:38.846984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38595c0-4177-4800-8139-bf0d2fb971b9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:38.847179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38595c0-4177-4800-8139-bf0d2fb971b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:38.865625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e38595c0-4177-4800-8139-bf0d2fb971b9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:38.866805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e38595c0-4177-4800-8139-bf0d2fb971b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:38.935857 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 645/1297] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:42:39.206328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72eb849-7265-4c9a-b687-5848878f242b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:39.303291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72eb849-7265-4c9a-b687-5848878f242b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:39.341520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafe68f-070f-4dfb-a625-fdfa7fc4cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:39.342152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafe68f-070f-4dfb-a625-fdfa7fc4cb48 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:39.342431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafe68f-070f-4dfb-a625-fdfa7fc4cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:39.342701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafe68f-070f-4dfb-a625-fdfa7fc4cb48 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:39.343270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafe68f-070f-4dfb-a625-fdfa7fc4cb48 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:39.344629 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafe68f-070f-4dfb-a625-fdfa7fc4cb48 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-2aafe68f-070f-4dfb-a625-fdfa7fc4cb48 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:39.353093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:39.354268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:39.355317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 653/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 3 14:42:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:39.360594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38595c0-4177-4800-8139-bf0d2fb971b9 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:39.377985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:39.456580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38595c0-4177-4800-8139-bf0d2fb971b9 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:39.488124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa02ad-d274-48ee-9113-d805c01c2fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:39.488713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa02ad-d274-48ee-9113-d805c01c2fed None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:39.488986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa02ad-d274-48ee-9113-d805c01c2fed None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:39.489250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa02ad-d274-48ee-9113-d805c01c2fed None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:39.489896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa02ad-d274-48ee-9113-d805c01c2fed None admin] RBAC: Adding query filter params (name=placement) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:39.490197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa02ad-d274-48ee-9113-d805c01c2fed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:39.491546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa02ad-d274-48ee-9113-d805c01c2fed None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:39.496882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 646/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Mon Oct 3 14:42:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:39.502823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafe68f-070f-4dfb-a625-fdfa7fc4cb48 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:39.533511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:39.600615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafe68f-070f-4dfb-a625-fdfa7fc4cb48 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:39.633103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c3548-4786-4118-a354-efef9c977d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:39.633715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c3548-4786-4118-a354-efef9c977d8f None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:39.633993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c3548-4786-4118-a354-efef9c977d8f None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:39.634262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c3548-4786-4118-a354-efef9c977d8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:39.634950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c3548-4786-4118-a354-efef9c977d8f None admin] RBAC: Adding query filter params (interface=public, service_id=b2c57e87a09a44ff89971306bd94dc0f, region_id=RegionOne) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:39.635257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c3548-4786-4118-a354-efef9c977d8f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:39.636614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c3548-4786-4118-a354-efef9c977d8f None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:39.641432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 654/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 3 14:42:39 2022] GET /identity/v3/endpoints?service_id=b2c57e87a09a44ff89971306bd94dc0f&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:39.647123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa02ad-d274-48ee-9113-d805c01c2fed None admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:39.723989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:39.742366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa02ad-d274-48ee-9113-d805c01c2fed None admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:39.773935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fbaa2-40f0-4634-9b19-b2826a029421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:39.774530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fbaa2-40f0-4634-9b19-b2826a029421 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:39.774804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fbaa2-40f0-4634-9b19-b2826a029421 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:39.775069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fbaa2-40f0-4634-9b19-b2826a029421 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:39.775666 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fbaa2-40f0-4634-9b19-b2826a029421 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:39.777550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fbaa2-40f0-4634-9b19-b2826a029421 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:39.784151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 647/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 3 14:42:39 2022] GET /identity/v3/services => generated 2191 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:42:41.917790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a614872-b57b-482e-ab25-496452b9f16a None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:41.918324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a614872-b57b-482e-ab25-496452b9f16a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:41.918324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a614872-b57b-482e-ab25-496452b9f16a None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:41.919678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 655/1302] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:42:41.924935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839282-5fd9-400c-8929-3d408164aceb None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:41.925325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839282-5fd9-400c-8929-3d408164aceb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:41.925483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839282-5fd9-400c-8929-3d408164aceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:41.943649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9b839282-5fd9-400c-8929-3d408164aceb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:42:41.944830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9b839282-5fd9-400c-8929-3d408164aceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:42:42.008873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 648/1303] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:42:41 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 14:42:42.279954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a614872-b57b-482e-ab25-496452b9f16a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:42.372819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:42.379469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a614872-b57b-482e-ab25-496452b9f16a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:42.411803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef6ff8-d8d7-42a4-a735-a34c7b6b4c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:42.412414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef6ff8-d8d7-42a4-a735-a34c7b6b4c84 None admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:42.412693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef6ff8-d8d7-42a4-a735-a34c7b6b4c84 None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:42.412966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef6ff8-d8d7-42a4-a735-a34c7b6b4c84 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:42.413566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef6ff8-d8d7-42a4-a735-a34c7b6b4c84 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:42.414924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef6ff8-d8d7-42a4-a735-a34c7b6b4c84 None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application [None req-27ef6ff8-d8d7-42a4-a735-a34c7b6b4c84 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 14:42:42.424043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 14:42:42.425186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application with Lock( Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self._enter() Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application return self.creator() Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application created_value = creator( Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: ERROR keystone.server.flask.application Oct 03 14:42:42.426514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 656/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 3 14:42:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 14:42:42.431057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b839282-5fd9-400c-8929-3d408164aceb None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:42.503027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:42.524784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b839282-5fd9-400c-8929-3d408164aceb None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:42.557253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42552a1b-1db6-41d4-b58d-4c7a8b1ea455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:42.557851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42552a1b-1db6-41d4-b58d-4c7a8b1ea455 None admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:42.558126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42552a1b-1db6-41d4-b58d-4c7a8b1ea455 None admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:42.558393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42552a1b-1db6-41d4-b58d-4c7a8b1ea455 None admin] PATH_INFO: `/v3/services` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:42.559020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42552a1b-1db6-41d4-b58d-4c7a8b1ea455 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:42:42.559319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42552a1b-1db6-41d4-b58d-4c7a8b1ea455 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:42.560662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42552a1b-1db6-41d4-b58d-4c7a8b1ea455 None admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:42.565485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 649/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Mon Oct 3 14:42:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:42:42.571075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef6ff8-d8d7-42a4-a735-a34c7b6b4c84 None admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:42:42.667187 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef6ff8-d8d7-42a4-a735-a34c7b6b4c84 None admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:42:42.698778 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6bf52-92ae-4d14-965f-3eb2b4d62d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:42:42.699354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6bf52-92ae-4d14-965f-3eb2b4d62d3d None admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:42:42.699628 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6bf52-92ae-4d14-965f-3eb2b4d62d3d None admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:42:42.699895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6bf52-92ae-4d14-965f-3eb2b4d62d3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:42:42.700543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6bf52-92ae-4d14-965f-3eb2b4d62d3d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:42:42.701928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6bf52-92ae-4d14-965f-3eb2b4d62d3d None admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:42:42.720641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 657/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:42:42 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:42:52.563166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=30 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:42:52.716557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:43:46.367860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc4d80-c6d4-47ac-8de7-a9f14a26216a None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:46.368317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc4d80-c6d4-47ac-8de7-a9f14a26216a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:46.368603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc4d80-c6d4-47ac-8de7-a9f14a26216a None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:46.375650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69114523-6dfd-46b2-bb35-3b38be65166f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:46.376065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69114523-6dfd-46b2-bb35-3b38be65166f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:46.376199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69114523-6dfd-46b2-bb35-3b38be65166f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:46.379648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 650/1307] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:43:46 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:43:46.480543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-69114523-6dfd-46b2-bb35-3b38be65166f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:43:46.482051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-69114523-6dfd-46b2-bb35-3b38be65166f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:46.693719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 658/1308] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:43:46.996531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65016388-012a-4238-9b1b-2d37e332a2a2 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:46.997208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65016388-012a-4238-9b1b-2d37e332a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:46.997740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65016388-012a-4238-9b1b-2d37e332a2a2 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:46.999615 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 651/1309] 10.222.0.46 () {54 vars in 964 bytes} [Mon Oct 3 14:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:43:47.006415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d50850-5ff1-4a44-be61-b5454cd4ede4 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:47.006798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d50850-5ff1-4a44-be61-b5454cd4ede4 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:47.007044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d50850-5ff1-4a44-be61-b5454cd4ede4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:47.096793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-59d50850-5ff1-4a44-be61-b5454cd4ede4 None None] MFA Rules not processed for user `05e9009410a54ce3ab88fa0cac87a7f9`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:43:47.098491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-59d50850-5ff1-4a44-be61-b5454cd4ede4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:47.123253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:43:47.187412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 659/1310] 10.222.0.46 () {58 vars in 1061 bytes} [Mon Oct 3 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:43:47.196651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65016388-012a-4238-9b1b-2d37e332a2a2 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:43:47.198929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-65016388-012a-4238-9b1b-2d37e332a2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:47.332818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65016388-012a-4238-9b1b-2d37e332a2a2 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:43:47.421742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807246c1-5464-4276-bcb2-70d5a1b8bbac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:43:47.422420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807246c1-5464-4276-bcb2-70d5a1b8bbac service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:47.422715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807246c1-5464-4276-bcb2-70d5a1b8bbac service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:47.422897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807246c1-5464-4276-bcb2-70d5a1b8bbac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:47.424701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-807246c1-5464-4276-bcb2-70d5a1b8bbac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:47.491613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:43:47.550866 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807246c1-5464-4276-bcb2-70d5a1b8bbac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:43:47.552229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807246c1-5464-4276-bcb2-70d5a1b8bbac service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:43:47.572539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 652/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:43:50.083054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42574d4d-5d04-4e3f-b9d4-edcac70a13db None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:50.083613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42574d4d-5d04-4e3f-b9d4-edcac70a13db None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:50.083613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42574d4d-5d04-4e3f-b9d4-edcac70a13db None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:50.085434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 660/1312] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:43:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:43:50.090711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de70d9f-a963-4790-9787-989a1a3ecb31 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:50.091106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de70d9f-a963-4790-9787-989a1a3ecb31 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:50.091269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de70d9f-a963-4790-9787-989a1a3ecb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:50.171759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9de70d9f-a963-4790-9787-989a1a3ecb31 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:43:50.172984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9de70d9f-a963-4790-9787-989a1a3ecb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:50.268693 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 653/1313] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:43:50.565996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42574d4d-5d04-4e3f-b9d4-edcac70a13db None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:43:50.567573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-42574d4d-5d04-4e3f-b9d4-edcac70a13db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:50.669734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42574d4d-5d04-4e3f-b9d4-edcac70a13db None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:43:50.741180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:43:50.743206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e9829-484d-45f9-89b2-f0c0736c39af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:43:50.744016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e9829-484d-45f9-89b2-f0c0736c39af service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:50.744309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e9829-484d-45f9-89b2-f0c0736c39af service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:50.744775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e9829-484d-45f9-89b2-f0c0736c39af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:50.746814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f06e9829-484d-45f9-89b2-f0c0736c39af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:50.852292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e9829-484d-45f9-89b2-f0c0736c39af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:43:50.853700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e9829-484d-45f9-89b2-f0c0736c39af service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:43:50.872239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 661/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:43:53.385668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198086ca-b19e-4719-ad96-3f0eecbc31d9 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:53.386244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198086ca-b19e-4719-ad96-3f0eecbc31d9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:53.386493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198086ca-b19e-4719-ad96-3f0eecbc31d9 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:53.388700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 654/1315] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:43:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:43:53.394617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4353c1c-1e04-4dc1-8293-4b6ce22b76dc None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:53.395016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4353c1c-1e04-4dc1-8293-4b6ce22b76dc None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:53.395591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4353c1c-1e04-4dc1-8293-4b6ce22b76dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:53.461060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e4353c1c-1e04-4dc1-8293-4b6ce22b76dc None None] MFA Rules not processed for user `0e508b7987364e59b4008c6b649a48e0`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:43:53.462285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e4353c1c-1e04-4dc1-8293-4b6ce22b76dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:53.512540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:43:53.556228 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 662/1316] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:43:53.852711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854da4ba-2013-448f-8411-ce1093bcd04b None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:53.853068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854da4ba-2013-448f-8411-ce1093bcd04b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:53.853347 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854da4ba-2013-448f-8411-ce1093bcd04b None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:53.854728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 655/1317] 10.222.0.46 () {54 vars in 964 bytes} [Mon Oct 3 14:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:43:53.860660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c134a59-c03c-491d-aade-a11dce6d82aa None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:53.860943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c134a59-c03c-491d-aade-a11dce6d82aa None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:53.861213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c134a59-c03c-491d-aade-a11dce6d82aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:53.936320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-1c134a59-c03c-491d-aade-a11dce6d82aa None None] MFA Rules not processed for user `05e9009410a54ce3ab88fa0cac87a7f9`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:43:53.937526 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1c134a59-c03c-491d-aade-a11dce6d82aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:54.010363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 663/1318] 10.222.0.46 () {58 vars in 1061 bytes} [Mon Oct 3 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:43:54.019284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854da4ba-2013-448f-8411-ce1093bcd04b None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:43:54.020820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-854da4ba-2013-448f-8411-ce1093bcd04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:54.030858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:43:54.122457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854da4ba-2013-448f-8411-ce1093bcd04b None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:43:54.193118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d68cc-1251-42be-b671-5ef4bb8dcc82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:43:54.194235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d68cc-1251-42be-b671-5ef4bb8dcc82 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:54.194518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d68cc-1251-42be-b671-5ef4bb8dcc82 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:54.194789 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d68cc-1251-42be-b671-5ef4bb8dcc82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:54.196364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-873d68cc-1251-42be-b671-5ef4bb8dcc82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:54.257127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:43:54.302942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d68cc-1251-42be-b671-5ef4bb8dcc82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:43:54.304947 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d68cc-1251-42be-b671-5ef4bb8dcc82 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:43:54.323025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 656/1319] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:43:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:43:57.375518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe5354-6e51-4c8d-b80f-6289d40d8936 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:57.376059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe5354-6e51-4c8d-b80f-6289d40d8936 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:57.376059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe5354-6e51-4c8d-b80f-6289d40d8936 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:57.377432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 664/1320] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:43:57.383035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff807a-44ad-4f32-b86c-cc33f3b5da6b None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:57.383448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff807a-44ad-4f32-b86c-cc33f3b5da6b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:57.383586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff807a-44ad-4f32-b86c-cc33f3b5da6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:57.447796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-72ff807a-44ad-4f32-b86c-cc33f3b5da6b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:43:57.449001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-72ff807a-44ad-4f32-b86c-cc33f3b5da6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:57.541425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 657/1321] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:43:57.821459 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe5354-6e51-4c8d-b80f-6289d40d8936 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:43:57.823053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3fbe5354-6e51-4c8d-b80f-6289d40d8936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:57.872486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:43:57.925456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe5354-6e51-4c8d-b80f-6289d40d8936 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:43:57.996686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a78f9a-aadf-40a0-99c4-c02083776b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:43:57.997311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a78f9a-aadf-40a0-99c4-c02083776b71 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:43:57.997613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a78f9a-aadf-40a0-99c4-c02083776b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:43:57.997876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a78f9a-aadf-40a0-99c4-c02083776b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:43:57.999434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-23a78f9a-aadf-40a0-99c4-c02083776b71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:43:58.095641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:43:58.108716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a78f9a-aadf-40a0-99c4-c02083776b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:43:58.110185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a78f9a-aadf-40a0-99c4-c02083776b71 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:43:58.127524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 665/1322] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:43:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:00.911192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdb3f5-a5e5-4063-9564-75ebbd9fa5f5 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:00.911777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdb3f5-a5e5-4063-9564-75ebbd9fa5f5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:00.911777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdb3f5-a5e5-4063-9564-75ebbd9fa5f5 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:00.913606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 658/1323] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:00.918968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bad19-ab7c-4811-8969-a541fe3fff69 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:00.919388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bad19-ab7c-4811-8969-a541fe3fff69 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:00.919520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bad19-ab7c-4811-8969-a541fe3fff69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:00.989829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e33bad19-ab7c-4811-8969-a541fe3fff69 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:00.991206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e33bad19-ab7c-4811-8969-a541fe3fff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:01.097567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 666/1324] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:01.380116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdb3f5-a5e5-4063-9564-75ebbd9fa5f5 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:01.381802 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-71bdb3f5-a5e5-4063-9564-75ebbd9fa5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:01.401756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:01.485375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdb3f5-a5e5-4063-9564-75ebbd9fa5f5 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:01.557071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc78d6-5ecc-48aa-91ce-0c176f87876a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:01.557701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc78d6-5ecc-48aa-91ce-0c176f87876a service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:01.557977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc78d6-5ecc-48aa-91ce-0c176f87876a service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:01.558245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc78d6-5ecc-48aa-91ce-0c176f87876a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:01.559837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ecbc78d6-5ecc-48aa-91ce-0c176f87876a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:01.630062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:01.667891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc78d6-5ecc-48aa-91ce-0c176f87876a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:01.669321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc78d6-5ecc-48aa-91ce-0c176f87876a service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:01.687092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 659/1325] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:03.885436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99551e6f-1c7c-40c4-ab47-2c756ec14f7e None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:03.885992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99551e6f-1c7c-40c4-ab47-2c756ec14f7e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:03.885992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99551e6f-1c7c-40c4-ab47-2c756ec14f7e None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:03.887362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 667/1326] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:03.892785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024df3da-f2a6-40e1-8bd2-91b562253fcb None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:03.893208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024df3da-f2a6-40e1-8bd2-91b562253fcb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:03.893336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024df3da-f2a6-40e1-8bd2-91b562253fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:03.958422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-024df3da-f2a6-40e1-8bd2-91b562253fcb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:03.960188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-024df3da-f2a6-40e1-8bd2-91b562253fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:04.064103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 660/1327] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:04.344216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99551e6f-1c7c-40c4-ab47-2c756ec14f7e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:04.345802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-99551e6f-1c7c-40c4-ab47-2c756ec14f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:04.406338 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:04.451893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99551e6f-1c7c-40c4-ab47-2c756ec14f7e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:04.524797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea48c1-7a45-4c74-8c65-0f3ca7435865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:04.525403 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea48c1-7a45-4c74-8c65-0f3ca7435865 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:04.525707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea48c1-7a45-4c74-8c65-0f3ca7435865 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:04.525971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea48c1-7a45-4c74-8c65-0f3ca7435865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:04.527564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2aea48c1-7a45-4c74-8c65-0f3ca7435865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:04.634435 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:04.636882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea48c1-7a45-4c74-8c65-0f3ca7435865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:04.642574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea48c1-7a45-4c74-8c65-0f3ca7435865 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:04.657653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 668/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:06.859772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e345c07-778d-4682-99cd-79f43f91ead8 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:06.860313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e345c07-778d-4682-99cd-79f43f91ead8 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:06.860313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e345c07-778d-4682-99cd-79f43f91ead8 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:06.861670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 661/1329] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:06.867631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e6d95-77b5-4e5f-8689-08b477c1e0f0 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:06.868047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e6d95-77b5-4e5f-8689-08b477c1e0f0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:06.868163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e6d95-77b5-4e5f-8689-08b477c1e0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:06.934827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e63e6d95-77b5-4e5f-8689-08b477c1e0f0 None None] MFA Rules not processed for user `0e508b7987364e59b4008c6b649a48e0`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:06.936023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e63e6d95-77b5-4e5f-8689-08b477c1e0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:07.030716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 669/1330] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:07.309988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e345c07-778d-4682-99cd-79f43f91ead8 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:07.311548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4e345c07-778d-4682-99cd-79f43f91ead8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:07.343885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:07.419683 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e345c07-778d-4682-99cd-79f43f91ead8 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:07.495038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13a9aa-a358-40c7-88a8-e7aaa140b3ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:07.495654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13a9aa-a358-40c7-88a8-e7aaa140b3ac service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:07.495931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13a9aa-a358-40c7-88a8-e7aaa140b3ac service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:07.496200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13a9aa-a358-40c7-88a8-e7aaa140b3ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:07.497833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7f13a9aa-a358-40c7-88a8-e7aaa140b3ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:07.576961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:07.609177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13a9aa-a358-40c7-88a8-e7aaa140b3ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:07.610673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13a9aa-a358-40c7-88a8-e7aaa140b3ac service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:07.629267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 662/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:10.067750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924993a7-9290-47c9-b933-a6a645358eae None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:10.068267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924993a7-9290-47c9-b933-a6a645358eae None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:10.068267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924993a7-9290-47c9-b933-a6a645358eae None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:10.069680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 670/1332] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:10.074764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6d042-a208-4cdb-8c1c-60d914fbea61 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:10.075166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6d042-a208-4cdb-8c1c-60d914fbea61 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:10.075305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6d042-a208-4cdb-8c1c-60d914fbea61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:10.139145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-64a6d042-a208-4cdb-8c1c-60d914fbea61 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:10.140330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-64a6d042-a208-4cdb-8c1c-60d914fbea61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:10.232936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 663/1333] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:10.516745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924993a7-9290-47c9-b933-a6a645358eae None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:10.518370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-924993a7-9290-47c9-b933-a6a645358eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:10.590577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:10.624511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924993a7-9290-47c9-b933-a6a645358eae None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:10.695766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fabb43-8847-4666-b978-e443739ee6c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:10.696369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fabb43-8847-4666-b978-e443739ee6c7 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:10.696646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fabb43-8847-4666-b978-e443739ee6c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:10.696918 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fabb43-8847-4666-b978-e443739ee6c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:10.698536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-35fabb43-8847-4666-b978-e443739ee6c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:10.804849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fabb43-8847-4666-b978-e443739ee6c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:10.806238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fabb43-8847-4666-b978-e443739ee6c7 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:10.815945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:10.824905 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 671/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:13.080006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dae092-f09e-482f-88ab-729f999131a2 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:13.080544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dae092-f09e-482f-88ab-729f999131a2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:13.080638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dae092-f09e-482f-88ab-729f999131a2 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:13.081961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 664/1335] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:13.087912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ef489-6996-4837-b0dd-95a83501cb2c None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:13.088325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ef489-6996-4837-b0dd-95a83501cb2c None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:13.088495 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ef489-6996-4837-b0dd-95a83501cb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:13.153981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-180ef489-6996-4837-b0dd-95a83501cb2c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:13.155180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-180ef489-6996-4837-b0dd-95a83501cb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:13.251261 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 672/1336] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:13.540351 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dae092-f09e-482f-88ab-729f999131a2 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:13.541927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-76dae092-f09e-482f-88ab-729f999131a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:13.581678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:13.644876 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dae092-f09e-482f-88ab-729f999131a2 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:13.716390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9ea82-d9be-46c6-acd6-9dfcaded1fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:13.717011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9ea82-d9be-46c6-acd6-9dfcaded1fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:13.717275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9ea82-d9be-46c6-acd6-9dfcaded1fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:13.717578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9ea82-d9be-46c6-acd6-9dfcaded1fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:13.719182 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ded9ea82-d9be-46c6-acd6-9dfcaded1fa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:13.799867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:13.825635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9ea82-d9be-46c6-acd6-9dfcaded1fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:13.827085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9ea82-d9be-46c6-acd6-9dfcaded1fa3 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:13.844974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 665/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:16.287493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549712d5-8bc4-4fee-a253-7d0c0af7b91d None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:16.288025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549712d5-8bc4-4fee-a253-7d0c0af7b91d None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:16.288025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549712d5-8bc4-4fee-a253-7d0c0af7b91d None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:16.289377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 673/1338] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:16.296004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e62967-d594-4901-ae57-d195b52e10e6 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:16.296385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e62967-d594-4901-ae57-d195b52e10e6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:16.296557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e62967-d594-4901-ae57-d195b52e10e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:16.360992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e9e62967-d594-4901-ae57-d195b52e10e6 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:16.362224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e9e62967-d594-4901-ae57-d195b52e10e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:16.457346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 666/1339] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:16.737068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549712d5-8bc4-4fee-a253-7d0c0af7b91d None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:16.738621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-549712d5-8bc4-4fee-a253-7d0c0af7b91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:16.812957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:16.839417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549712d5-8bc4-4fee-a253-7d0c0af7b91d None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:16.910743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6a91d-d6b8-421c-85e6-502920f248e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:16.911923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6a91d-d6b8-421c-85e6-502920f248e5 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:16.912195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6a91d-d6b8-421c-85e6-502920f248e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:16.912457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6a91d-d6b8-421c-85e6-502920f248e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:16.914049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b1d6a91d-d6b8-421c-85e6-502920f248e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:17.018811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6a91d-d6b8-421c-85e6-502920f248e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:17.020239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6a91d-d6b8-421c-85e6-502920f248e5 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:17.039666 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 674/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:21.041183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079a9ae-2c98-4e15-b00d-b74ed9c4bdc9 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:21.041798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079a9ae-2c98-4e15-b00d-b74ed9c4bdc9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:21.041798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079a9ae-2c98-4e15-b00d-b74ed9c4bdc9 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:21.043141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 667/1341] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:21.048569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d9173-8577-40d4-8d88-8eebb96f54e1 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:21.048992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d9173-8577-40d4-8d88-8eebb96f54e1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:21.049238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d9173-8577-40d4-8d88-8eebb96f54e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:21.054780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:21.116086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-cd7d9173-8577-40d4-8d88-8eebb96f54e1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:21.117201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-cd7d9173-8577-40d4-8d88-8eebb96f54e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:21.211078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 675/1342] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:21.493831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079a9ae-2c98-4e15-b00d-b74ed9c4bdc9 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:21.495362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a079a9ae-2c98-4e15-b00d-b74ed9c4bdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:21.544101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:21.601437 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079a9ae-2c98-4e15-b00d-b74ed9c4bdc9 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:21.675927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f78d8e-990c-473d-8eb7-cd02c91ddd11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:21.677052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f78d8e-990c-473d-8eb7-cd02c91ddd11 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:21.677334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f78d8e-990c-473d-8eb7-cd02c91ddd11 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:21.677635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f78d8e-990c-473d-8eb7-cd02c91ddd11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:21.679240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-46f78d8e-990c-473d-8eb7-cd02c91ddd11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:21.772620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:21.789386 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f78d8e-990c-473d-8eb7-cd02c91ddd11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:21.790918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f78d8e-990c-473d-8eb7-cd02c91ddd11 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:21.809232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 668/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:24.672108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8d188-19ef-4b1e-9c8c-224ae9dfee11 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:24.672672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8d188-19ef-4b1e-9c8c-224ae9dfee11 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:24.672672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8d188-19ef-4b1e-9c8c-224ae9dfee11 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:24.674047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 676/1344] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:24.679879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403c197-2a70-4a2d-9775-1e01f29f3412 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:24.680285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403c197-2a70-4a2d-9775-1e01f29f3412 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:24.680436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403c197-2a70-4a2d-9775-1e01f29f3412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:24.747457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a403c197-2a70-4a2d-9775-1e01f29f3412 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:24.748680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a403c197-2a70-4a2d-9775-1e01f29f3412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:24.850417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 669/1345] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:25.146894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8d188-19ef-4b1e-9c8c-224ae9dfee11 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:25.148427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b1d8d188-19ef-4b1e-9c8c-224ae9dfee11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:25.236339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:25.253137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8d188-19ef-4b1e-9c8c-224ae9dfee11 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:25.330379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d1e40-d968-4211-ab0e-75120667155c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:25.330982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d1e40-d968-4211-ab0e-75120667155c service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:25.331254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d1e40-d968-4211-ab0e-75120667155c service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:25.331642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d1e40-d968-4211-ab0e-75120667155c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:25.333858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1a4d1e40-d968-4211-ab0e-75120667155c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:25.442213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d1e40-d968-4211-ab0e-75120667155c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:25.443635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d1e40-d968-4211-ab0e-75120667155c service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:25.461488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 677/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:29.564976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a25e5f-5306-4e4e-9365-8fe565377b2c None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:29.565529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a25e5f-5306-4e4e-9365-8fe565377b2c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:29.565529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a25e5f-5306-4e4e-9365-8fe565377b2c None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:29.566931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 670/1347] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:29.572182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f30e8c-9a7a-4e3a-a70a-cb634549fb7d None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:29.572566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f30e8c-9a7a-4e3a-a70a-cb634549fb7d None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:29.572761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f30e8c-9a7a-4e3a-a70a-cb634549fb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:29.588777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:29.642794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-23f30e8c-9a7a-4e3a-a70a-cb634549fb7d None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:29.644008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-23f30e8c-9a7a-4e3a-a70a-cb634549fb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:29.741250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 678/1348] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:30.022986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a25e5f-5306-4e4e-9365-8fe565377b2c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:30.024544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-23a25e5f-5306-4e4e-9365-8fe565377b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:30.087409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:30.134474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a25e5f-5306-4e4e-9365-8fe565377b2c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:30.206079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65533c-fff7-4a3b-a15c-1cecc1b1386e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:30.206679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65533c-fff7-4a3b-a15c-1cecc1b1386e service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:30.206951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65533c-fff7-4a3b-a15c-1cecc1b1386e service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:30.207219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65533c-fff7-4a3b-a15c-1cecc1b1386e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:30.208839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8b65533c-fff7-4a3b-a15c-1cecc1b1386e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:30.315230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:30.319337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65533c-fff7-4a3b-a15c-1cecc1b1386e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:30.320767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65533c-fff7-4a3b-a15c-1cecc1b1386e service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:30.338460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 671/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:35.232464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d1bcc-5c92-491c-9551-4b40c373314f None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:35.232464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d1bcc-5c92-491c-9551-4b40c373314f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:35.232464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d1bcc-5c92-491c-9551-4b40c373314f None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:35.234004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 679/1350] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:44:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:35.239742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068753d7-375e-44e7-bb97-ceb34a112f30 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:35.240760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068753d7-375e-44e7-bb97-ceb34a112f30 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:35.241260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068753d7-375e-44e7-bb97-ceb34a112f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:35.336906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-068753d7-375e-44e7-bb97-ceb34a112f30 None None] MFA Rules not processed for user `ebc01c87551044818e1ccbb46b8e7340`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:35.338535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-068753d7-375e-44e7-bb97-ceb34a112f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:35.524608 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 672/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:39.558504 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57652500-a7b7-41a4-b117-c72b525425ce None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:39.559009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57652500-a7b7-41a4-b117-c72b525425ce None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:39.559123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57652500-a7b7-41a4-b117-c72b525425ce None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:39.560657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 680/1352] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:39.565429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c808b6f-ac3b-4171-b058-919452dbb4cc None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:39.565890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c808b6f-ac3b-4171-b058-919452dbb4cc None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:39.566008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c808b6f-ac3b-4171-b058-919452dbb4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:39.623700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:39.648225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6c808b6f-ac3b-4171-b058-919452dbb4cc None None] MFA Rules not processed for user `ebc01c87551044818e1ccbb46b8e7340`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:39.649444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6c808b6f-ac3b-4171-b058-919452dbb4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:39.739628 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=59 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:39.764184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 673/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:41.214468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e940f4-f922-412b-86d4-cc129ab2794b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:41.215001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e940f4-f922-412b-86d4-cc129ab2794b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:41.215001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e940f4-f922-412b-86d4-cc129ab2794b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:41.216345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 681/1354] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:41.221263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee1cb-188e-4a16-b3bb-d2d98d8284f0 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:41.221729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee1cb-188e-4a16-b3bb-d2d98d8284f0 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:41.222011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee1cb-188e-4a16-b3bb-d2d98d8284f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:41.301908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-52fee1cb-188e-4a16-b3bb-d2d98d8284f0 None None] MFA Rules not processed for user `ebc01c87551044818e1ccbb46b8e7340`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:41.303139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-52fee1cb-188e-4a16-b3bb-d2d98d8284f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:41.344480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:41.411077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 674/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:44:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:45.065467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f825286-a3ae-4010-8ed6-b6eac4f32d9b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:45.066569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f825286-a3ae-4010-8ed6-b6eac4f32d9b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:45.066569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f825286-a3ae-4010-8ed6-b6eac4f32d9b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:45.067815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 682/1356] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:45.073291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c85293-874d-46c6-9692-d89983ee959b None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:45.073817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c85293-874d-46c6-9692-d89983ee959b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:45.074265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c85293-874d-46c6-9692-d89983ee959b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:45.145263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f5c85293-874d-46c6-9692-d89983ee959b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:45.146493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f5c85293-874d-46c6-9692-d89983ee959b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:45.247391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 675/1357] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:45.543655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b00b7-1cda-4531-9051-9c95afe97293 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:45.544166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b00b7-1cda-4531-9051-9c95afe97293 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:45.544639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b00b7-1cda-4531-9051-9c95afe97293 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:45.546248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 683/1358] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 3 14:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:45.555042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7f216-f1f0-4deb-b457-f038801be616 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:45.555406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7f216-f1f0-4deb-b457-f038801be616 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:45.555689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7f216-f1f0-4deb-b457-f038801be616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:45.573121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:45.636410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-17a7f216-f1f0-4deb-b457-f038801be616 None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:45.637615 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-17a7f216-f1f0-4deb-b457-f038801be616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:45.765473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 676/1359] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 3 14:44:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:44:45.773889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b00b7-1cda-4531-9051-9c95afe97293 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:45.775688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-840b00b7-1cda-4531-9051-9c95afe97293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:45.933647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b00b7-1cda-4531-9051-9c95afe97293 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:46.009152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:46.052329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39d8a9-1638-4145-a8c7-d83e6edd5be3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:46.052932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39d8a9-1638-4145-a8c7-d83e6edd5be3 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:46.053205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39d8a9-1638-4145-a8c7-d83e6edd5be3 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:46.053471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39d8a9-1638-4145-a8c7-d83e6edd5be3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:46.055089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5a39d8a9-1638-4145-a8c7-d83e6edd5be3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:46.151150 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39d8a9-1638-4145-a8c7-d83e6edd5be3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:46.152799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39d8a9-1638-4145-a8c7-d83e6edd5be3 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:46.172763 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 684/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:44:48.554853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c955a2-9e2b-45c8-9e3a-598762f4fcbc None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:48.555334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c955a2-9e2b-45c8-9e3a-598762f4fcbc None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:48.555334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c955a2-9e2b-45c8-9e3a-598762f4fcbc None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:48.558272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 677/1361] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:48.568219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096709a-ae4d-4551-8853-58f69cd8b84c None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:48.568219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096709a-ae4d-4551-8853-58f69cd8b84c None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:48.568803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096709a-ae4d-4551-8853-58f69cd8b84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:48.647518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-0096709a-ae4d-4551-8853-58f69cd8b84c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:48.648904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-0096709a-ae4d-4551-8853-58f69cd8b84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:48.687719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:48.749341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 685/1362] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:49.037837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c955a2-9e2b-45c8-9e3a-598762f4fcbc None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:49.039561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-04c955a2-9e2b-45c8-9e3a-598762f4fcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:49.107069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:49.210711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c955a2-9e2b-45c8-9e3a-598762f4fcbc None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:49.337844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bcabf-24e8-4473-a738-a989d692b15b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:49.338464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bcabf-24e8-4473-a738-a989d692b15b service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:49.338772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bcabf-24e8-4473-a738-a989d692b15b service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:49.339066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bcabf-24e8-4473-a738-a989d692b15b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:49.340850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5d5bcabf-24e8-4473-a738-a989d692b15b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:49.365271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:49.433325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bcabf-24e8-4473-a738-a989d692b15b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:49.434971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bcabf-24e8-4473-a738-a989d692b15b service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:49.453814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 678/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:51.931676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6189e12-3ba5-4200-85b2-557cab94c058 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:51.932274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6189e12-3ba5-4200-85b2-557cab94c058 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:51.932274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6189e12-3ba5-4200-85b2-557cab94c058 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:51.934076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 686/1364] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:51.939746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dec065-a045-4d93-b194-786561727629 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:51.940236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dec065-a045-4d93-b194-786561727629 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:51.940327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dec065-a045-4d93-b194-786561727629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:52.009688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b2dec065-a045-4d93-b194-786561727629 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:52.010914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b2dec065-a045-4d93-b194-786561727629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:52.108565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:52.119940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 679/1365] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:52.402757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a5561-fa6c-4f71-a10b-cbe5aaf15b8b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:52.403158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a5561-fa6c-4f71-a10b-cbe5aaf15b8b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:52.403428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a5561-fa6c-4f71-a10b-cbe5aaf15b8b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:52.404953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 687/1366] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 3 14:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:52.410970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d19aba-f1d5-41e0-a752-a9361ebdc06a None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:52.411304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d19aba-f1d5-41e0-a752-a9361ebdc06a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:52.411585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d19aba-f1d5-41e0-a752-a9361ebdc06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:52.491130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f8d19aba-f1d5-41e0-a752-a9361ebdc06a None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:52.492358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f8d19aba-f1d5-41e0-a752-a9361ebdc06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:52.621572 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 680/1367] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 3 14:44:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:52.630290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a5561-fa6c-4f71-a10b-cbe5aaf15b8b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:52.631880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-fe2a5561-fa6c-4f71-a10b-cbe5aaf15b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:52.785996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a5561-fa6c-4f71-a10b-cbe5aaf15b8b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:52.807180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:52.907675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936751df-b9f6-4d87-8526-7e6ed2adc4da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:52.908545 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936751df-b9f6-4d87-8526-7e6ed2adc4da service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:52.908921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936751df-b9f6-4d87-8526-7e6ed2adc4da service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:52.909325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936751df-b9f6-4d87-8526-7e6ed2adc4da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:52.911056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-936751df-b9f6-4d87-8526-7e6ed2adc4da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:53.005945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936751df-b9f6-4d87-8526-7e6ed2adc4da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:53.007716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936751df-b9f6-4d87-8526-7e6ed2adc4da service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:53.033539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 688/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:44:53.241846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa91e00-0751-43f2-a19e-ea177eed36c2 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:53.242380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa91e00-0751-43f2-a19e-ea177eed36c2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:53.242380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa91e00-0751-43f2-a19e-ea177eed36c2 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:53.243917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 681/1369] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:53.249516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8fb50-aebd-44b4-b677-9836dd7fa846 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:53.249820 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8fb50-aebd-44b4-b677-9836dd7fa846 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:53.250092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8fb50-aebd-44b4-b677-9836dd7fa846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:53.294098 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:53.338985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ace8fb50-aebd-44b4-b677-9836dd7fa846 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:53.341034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ace8fb50-aebd-44b4-b677-9836dd7fa846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:53.448172 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 689/1370] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:53.809358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59882931-e32f-4a49-abe6-a555785eceeb None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:53.809678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59882931-e32f-4a49-abe6-a555785eceeb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:53.809956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59882931-e32f-4a49-abe6-a555785eceeb None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:53.811469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 682/1371] 10.222.0.46 () {54 vars in 963 bytes} [Mon Oct 3 14:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:44:53.816763 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c974b5e-c3a3-4415-a736-264b91c69d80 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:53.817045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c974b5e-c3a3-4415-a736-264b91c69d80 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:53.817318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c974b5e-c3a3-4415-a736-264b91c69d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:53.901659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5c974b5e-c3a3-4415-a736-264b91c69d80 None None] MFA Rules not processed for user `64a9421bec3b4b9e89dec35d7556d025`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:53.902912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5c974b5e-c3a3-4415-a736-264b91c69d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:53.953419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:53.986547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 690/1372] 10.222.0.46 () {58 vars in 1060 bytes} [Mon Oct 3 14:44:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:53.994613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59882931-e32f-4a49-abe6-a555785eceeb None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:53.996258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-59882931-e32f-4a49-abe6-a555785eceeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:54.053262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:54.115544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59882931-e32f-4a49-abe6-a555785eceeb None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:54.201562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca74bd9-8695-405e-901f-b7cef489968c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:54.202165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca74bd9-8695-405e-901f-b7cef489968c service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:54.202457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca74bd9-8695-405e-901f-b7cef489968c service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:54.202705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca74bd9-8695-405e-901f-b7cef489968c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:54.204344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7ca74bd9-8695-405e-901f-b7cef489968c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:54.319171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:54.357052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca74bd9-8695-405e-901f-b7cef489968c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:54.361178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca74bd9-8695-405e-901f-b7cef489968c service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:54.389971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 683/1373] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:55.277895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1df786-0734-491d-8044-e54d0c7b5162 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:55.278433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1df786-0734-491d-8044-e54d0c7b5162 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:55.278433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1df786-0734-491d-8044-e54d0c7b5162 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:55.279869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 691/1374] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:55.285656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0400c-7bee-4b66-ae7b-7bdc44a78d65 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:55.286137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0400c-7bee-4b66-ae7b-7bdc44a78d65 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:55.286421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0400c-7bee-4b66-ae7b-7bdc44a78d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:55.353393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-c0c0400c-7bee-4b66-ae7b-7bdc44a78d65 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:55.354653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c0c0400c-7bee-4b66-ae7b-7bdc44a78d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:55.449181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 684/1375] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:44:55.740120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1df786-0734-491d-8044-e54d0c7b5162 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:55.741867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bb1df786-0734-491d-8044-e54d0c7b5162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:55.895764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1df786-0734-491d-8044-e54d0c7b5162 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:55.958855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:56.027111 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af091887-64f3-4ccd-8b05-9e45a0472fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:56.027770 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af091887-64f3-4ccd-8b05-9e45a0472fef service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:56.028357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af091887-64f3-4ccd-8b05-9e45a0472fef service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:56.028625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af091887-64f3-4ccd-8b05-9e45a0472fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:56.030395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-af091887-64f3-4ccd-8b05-9e45a0472fef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:56.129120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af091887-64f3-4ccd-8b05-9e45a0472fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:56.130775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af091887-64f3-4ccd-8b05-9e45a0472fef service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:56.149528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 692/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:44:58.360326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb2231-c242-433e-9b78-3a9e50334d48 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:58.360326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb2231-c242-433e-9b78-3a9e50334d48 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:58.361228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb2231-c242-433e-9b78-3a9e50334d48 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:58.365919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 685/1377] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:44:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:44:58.367888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867c861-ea45-4d00-90ac-4948b42994e0 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:58.368309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867c861-ea45-4d00-90ac-4948b42994e0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:58.368495 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867c861-ea45-4d00-90ac-4948b42994e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:58.440717 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c867c861-ea45-4d00-90ac-4948b42994e0 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:44:58.442491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c867c861-ea45-4d00-90ac-4948b42994e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:58.450329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:58.549557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 693/1378] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:44:58.834746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb2231-c242-433e-9b78-3a9e50334d48 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:44:58.836266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-fedb2231-c242-433e-9b78-3a9e50334d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:58.900224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:58.989769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb2231-c242-433e-9b78-3a9e50334d48 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:44:59.117994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac781a-5729-4800-a72a-592369530862 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:44:59.118527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac781a-5729-4800-a72a-592369530862 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:44:59.118803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac781a-5729-4800-a72a-592369530862 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:44:59.119076 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac781a-5729-4800-a72a-592369530862 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:44:59.120660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5fac781a-5729-4800-a72a-592369530862 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:44:59.140241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:44:59.213858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac781a-5729-4800-a72a-592369530862 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:44:59.215471 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac781a-5729-4800-a72a-592369530862 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:44:59.234044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 686/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:44:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:01.448568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e5d89-3126-4a71-8c6a-dd220ebe5f9e None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:01.449158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e5d89-3126-4a71-8c6a-dd220ebe5f9e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:01.449158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e5d89-3126-4a71-8c6a-dd220ebe5f9e None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:01.450662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 694/1380] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:01.455686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116a2b-511b-4700-ba86-75821c69c160 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:01.456089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116a2b-511b-4700-ba86-75821c69c160 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:01.456312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116a2b-511b-4700-ba86-75821c69c160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:01.523144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ad116a2b-511b-4700-ba86-75821c69c160 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:01.524353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ad116a2b-511b-4700-ba86-75821c69c160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:01.618644 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:01.625778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 687/1381] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:01.916658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e5d89-3126-4a71-8c6a-dd220ebe5f9e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:01.918215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-974e5d89-3126-4a71-8c6a-dd220ebe5f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:02.058028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:02.068710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e5d89-3126-4a71-8c6a-dd220ebe5f9e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:02.195290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dde7c5-3ab7-432c-877f-ddfcc0c3c4f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:02.196121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde7c5-3ab7-432c-877f-ddfcc0c3c4f3 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:02.196463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde7c5-3ab7-432c-877f-ddfcc0c3c4f3 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:02.196810 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde7c5-3ab7-432c-877f-ddfcc0c3c4f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:02.198879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d0dde7c5-3ab7-432c-877f-ddfcc0c3c4f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:02.294237 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dde7c5-3ab7-432c-877f-ddfcc0c3c4f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:02.296006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dde7c5-3ab7-432c-877f-ddfcc0c3c4f3 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:02.306708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:02.315302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 695/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:04.962350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb4a73-b8e4-4cc4-a72c-197e41606d5c None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:04.963064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb4a73-b8e4-4cc4-a72c-197e41606d5c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:04.963064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb4a73-b8e4-4cc4-a72c-197e41606d5c None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:04.964201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 688/1383] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:04.969734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31086013-a125-430a-be6b-f3414067ed3c None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:04.970240 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31086013-a125-430a-be6b-f3414067ed3c None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:04.970240 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31086013-a125-430a-be6b-f3414067ed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:05.038526 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-31086013-a125-430a-be6b-f3414067ed3c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:05.039790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-31086013-a125-430a-be6b-f3414067ed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:05.150499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 696/1384] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:05.435754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb4a73-b8e4-4cc4-a72c-197e41606d5c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:05.437325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-eebb4a73-b8e4-4cc4-a72c-197e41606d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:05.595474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb4a73-b8e4-4cc4-a72c-197e41606d5c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:05.673323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:05.724961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f0a9c-4820-4e2c-bb80-0c673e7a739a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:05.725889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f0a9c-4820-4e2c-bb80-0c673e7a739a service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:05.726235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f0a9c-4820-4e2c-bb80-0c673e7a739a service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:05.726582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f0a9c-4820-4e2c-bb80-0c673e7a739a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:05.728635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8a9f0a9c-4820-4e2c-bb80-0c673e7a739a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:05.829168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f0a9c-4820-4e2c-bb80-0c673e7a739a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:05.831933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f0a9c-4820-4e2c-bb80-0c673e7a739a service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:05.852588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 689/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:08.023033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9edb5-3163-4bd4-8c45-8ab543480827 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:08.023591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9edb5-3163-4bd4-8c45-8ab543480827 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:08.023591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9edb5-3163-4bd4-8c45-8ab543480827 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:08.025181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 697/1386] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:08.030482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec63071-d55f-48b9-ac67-9f50d92abfe9 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:08.030932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec63071-d55f-48b9-ac67-9f50d92abfe9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:08.031036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec63071-d55f-48b9-ac67-9f50d92abfe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:08.098085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-7ec63071-d55f-48b9-ac67-9f50d92abfe9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:08.099221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7ec63071-d55f-48b9-ac67-9f50d92abfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:08.131996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:08.196207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 690/1387] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:45:08.485281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9edb5-3163-4bd4-8c45-8ab543480827 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:08.486931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-43d9edb5-3163-4bd4-8c45-8ab543480827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:08.562802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:08.637518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9edb5-3163-4bd4-8c45-8ab543480827 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:08.769692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414785f-8ddc-4138-86e2-52b311f56adb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:08.770345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414785f-8ddc-4138-86e2-52b311f56adb service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:08.770691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414785f-8ddc-4138-86e2-52b311f56adb service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:08.771027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414785f-8ddc-4138-86e2-52b311f56adb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:08.772983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8414785f-8ddc-4138-86e2-52b311f56adb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:08.819092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:08.875977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414785f-8ddc-4138-86e2-52b311f56adb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:08.878158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414785f-8ddc-4138-86e2-52b311f56adb service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:08.895860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 698/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:09.891981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169a186-86df-4322-9582-3500e1a0c483 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:09.892524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169a186-86df-4322-9582-3500e1a0c483 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:09.893207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169a186-86df-4322-9582-3500e1a0c483 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:09.894585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 691/1389] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:09.899902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96591bfc-042f-406a-af2b-3dae1125c8f1 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:09.900447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96591bfc-042f-406a-af2b-3dae1125c8f1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:09.900562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96591bfc-042f-406a-af2b-3dae1125c8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:09.972944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-96591bfc-042f-406a-af2b-3dae1125c8f1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:09.974169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-96591bfc-042f-406a-af2b-3dae1125c8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:10.073182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 699/1390] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:10.393974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2493efc-714a-4170-af9b-b2d33e51222b None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:10.394354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2493efc-714a-4170-af9b-b2d33e51222b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:10.394698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2493efc-714a-4170-af9b-b2d33e51222b None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:10.396248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 692/1391] 10.222.0.46 () {54 vars in 962 bytes} [Mon Oct 3 14:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:10.401802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb1cb8-a01a-4d22-87a7-1ce0bd852e94 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:10.402092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb1cb8-a01a-4d22-87a7-1ce0bd852e94 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:10.402366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb1cb8-a01a-4d22-87a7-1ce0bd852e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:10.418329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:10.481017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-afeb1cb8-a01a-4d22-87a7-1ce0bd852e94 None None] MFA Rules not processed for user `dd04df68f9d84fc4bb981c8b69cc771a`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:10.482246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-afeb1cb8-a01a-4d22-87a7-1ce0bd852e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:10.560138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 700/1392] 10.222.0.46 () {58 vars in 1059 bytes} [Mon Oct 3 14:45:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:10.569030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2493efc-714a-4170-af9b-b2d33e51222b None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:10.570625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a2493efc-714a-4170-af9b-b2d33e51222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:10.675345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2493efc-714a-4170-af9b-b2d33e51222b None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:10.734906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:10.751369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad621b1-60d7-40f5-ab10-4ad7a23b2536 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd04df68f9d84fc4bb981c8b69cc771a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:10.751976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad621b1-60d7-40f5-ab10-4ad7a23b2536 service glance] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:10.752251 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad621b1-60d7-40f5-ab10-4ad7a23b2536 service glance] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:10.752518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad621b1-60d7-40f5-ab10-4ad7a23b2536 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:10.754126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5ad621b1-60d7-40f5-ab10-4ad7a23b2536 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:10.864471 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad621b1-60d7-40f5-ab10-4ad7a23b2536 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:10.865888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad621b1-60d7-40f5-ab10-4ad7a23b2536 service glance] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:10.884119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 693/1393] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:11.068080 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30a3bf-c368-4737-8179-74659b10e2b2 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:11.068627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30a3bf-c368-4737-8179-74659b10e2b2 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:11.068722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30a3bf-c368-4737-8179-74659b10e2b2 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:11.070415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 701/1394] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:11.076699 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc358a7-9374-4ff4-b417-3e0c94fda0f9 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:11.077247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc358a7-9374-4ff4-b417-3e0c94fda0f9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:11.077247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc358a7-9374-4ff4-b417-3e0c94fda0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:11.153945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2bc358a7-9374-4ff4-b417-3e0c94fda0f9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:11.155268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2bc358a7-9374-4ff4-b417-3e0c94fda0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:11.199571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:11.261960 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 694/1395] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:11.551790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30a3bf-c368-4737-8179-74659b10e2b2 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:11.553404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ea30a3bf-c368-4737-8179-74659b10e2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:11.659567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c52f90-a32c-4fe7-bff9-6b7e2af51061 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:11.659846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:11.660219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c52f90-a32c-4fe7-bff9-6b7e2af51061 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:11.660720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c52f90-a32c-4fe7-bff9-6b7e2af51061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:11.706477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30a3bf-c368-4737-8179-74659b10e2b2 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:11.728753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-77c52f90-a32c-4fe7-bff9-6b7e2af51061 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:11.730315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-77c52f90-a32c-4fe7-bff9-6b7e2af51061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:11.818564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 695/1396] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:45:11.824089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356211f-60ae-4cee-af86-6e265f3818af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:11.824693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356211f-60ae-4cee-af86-6e265f3818af service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:11.824964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356211f-60ae-4cee-af86-6e265f3818af service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:11.825230 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356211f-60ae-4cee-af86-6e265f3818af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:11.826922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e356211f-60ae-4cee-af86-6e265f3818af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:11.865917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d37c1-544b-4660-9f60-f542453eb3b6 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:11.866208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d37c1-544b-4660-9f60-f542453eb3b6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:11.866487 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d37c1-544b-4660-9f60-f542453eb3b6 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:11.867829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 696/1397] 10.222.0.46 () {54 vars in 943 bytes} [Mon Oct 3 14:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:45:11.873799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9046bd-ac28-4855-9dcb-4d70824f46b9 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:11.874084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9046bd-ac28-4855-9dcb-4d70824f46b9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:11.874378 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9046bd-ac28-4855-9dcb-4d70824f46b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:11.900626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:11.917968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:11.931878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356211f-60ae-4cee-af86-6e265f3818af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:11.937523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356211f-60ae-4cee-af86-6e265f3818af service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:11.956753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 702/1398] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:11.960271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9e9046bd-ac28-4855-9dcb-4d70824f46b9 None None] MFA Rules not processed for user `6fbf23ae20ee4f6c979e256d2d4daa5f`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:11.962212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9e9046bd-ac28-4855-9dcb-4d70824f46b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:12.098330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 697/1399] 10.222.0.46 () {58 vars in 1040 bytes} [Mon Oct 3 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:45:12.110756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356211f-60ae-4cee-af86-6e265f3818af service nova] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:12.112674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e356211f-60ae-4cee-af86-6e265f3818af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:12.209682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356211f-60ae-4cee-af86-6e265f3818af service nova] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:12.314151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:12.316039 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded49df0-fad0-4a89-a169-0176d126470c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:12.316718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded49df0-fad0-4a89-a169-0176d126470c service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:12.317171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded49df0-fad0-4a89-a169-0176d126470c service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:12.317638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded49df0-fad0-4a89-a169-0176d126470c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:12.319433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ded49df0-fad0-4a89-a169-0176d126470c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:12.403096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded49df0-fad0-4a89-a169-0176d126470c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:12.404692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded49df0-fad0-4a89-a169-0176d126470c service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:12.413146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 703/1400] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 14:45:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 14:45:14.243166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bffcd5-65b4-4bd0-a524-c778d15518d7 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:14.243808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bffcd5-65b4-4bd0-a524-c778d15518d7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:14.243808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bffcd5-65b4-4bd0-a524-c778d15518d7 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:14.245037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 698/1401] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:14.250474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b280ea7-5767-4ce3-b3ba-5b08b36d7515 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:14.251292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b280ea7-5767-4ce3-b3ba-5b08b36d7515 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:14.251605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b280ea7-5767-4ce3-b3ba-5b08b36d7515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:14.328777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5b280ea7-5767-4ce3-b3ba-5b08b36d7515 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:14.330660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5b280ea7-5767-4ce3-b3ba-5b08b36d7515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:14.434489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:14.442749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 704/1402] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:14.727603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bffcd5-65b4-4bd0-a524-c778d15518d7 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:14.729138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-21bffcd5-65b4-4bd0-a524-c778d15518d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:14.829067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:14.899102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bffcd5-65b4-4bd0-a524-c778d15518d7 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:15.021936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee47db-929d-4d09-bee1-28017a7a1f6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:15.022476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee47db-929d-4d09-bee1-28017a7a1f6e service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:15.022756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee47db-929d-4d09-bee1-28017a7a1f6e service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:15.023160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee47db-929d-4d09-bee1-28017a7a1f6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:15.024821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-60ee47db-929d-4d09-bee1-28017a7a1f6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:15.086035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:15.134084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee47db-929d-4d09-bee1-28017a7a1f6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:15.135735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee47db-929d-4d09-bee1-28017a7a1f6e service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:15.156401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 699/1403] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:17.406714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2447908-746f-4fb1-95db-24accd5da9a1 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:17.407319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2447908-746f-4fb1-95db-24accd5da9a1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:17.407319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2447908-746f-4fb1-95db-24accd5da9a1 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:17.409147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 705/1404] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:45:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:17.414524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d4aee-80fe-4e67-a719-09910881c041 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:17.414964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d4aee-80fe-4e67-a719-09910881c041 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:17.415054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d4aee-80fe-4e67-a719-09910881c041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:17.494545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-311d4aee-80fe-4e67-a719-09910881c041 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:17.495789 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-311d4aee-80fe-4e67-a719-09910881c041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:17.604741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 700/1405] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:45:17.913295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2447908-746f-4fb1-95db-24accd5da9a1 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:17.915097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d2447908-746f-4fb1-95db-24accd5da9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:18.086646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2447908-746f-4fb1-95db-24accd5da9a1 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:18.162210 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:18.217296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea653b82-fa02-463d-90fe-7ea05414fb33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:18.217932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea653b82-fa02-463d-90fe-7ea05414fb33 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:18.218209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea653b82-fa02-463d-90fe-7ea05414fb33 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:18.221567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea653b82-fa02-463d-90fe-7ea05414fb33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:18.221567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ea653b82-fa02-463d-90fe-7ea05414fb33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:18.331302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea653b82-fa02-463d-90fe-7ea05414fb33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:18.333072 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea653b82-fa02-463d-90fe-7ea05414fb33 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:18.354643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 706/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:20.607117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdd44a-26e9-47b2-9b23-05d4ddb78b46 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:20.607647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdd44a-26e9-47b2-9b23-05d4ddb78b46 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:20.607647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdd44a-26e9-47b2-9b23-05d4ddb78b46 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:20.608924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 701/1407] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:20.614422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488fe34-f306-4b11-a5db-c8dfb85cd742 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:20.614878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488fe34-f306-4b11-a5db-c8dfb85cd742 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:20.614878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488fe34-f306-4b11-a5db-c8dfb85cd742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:20.686192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e488fe34-f306-4b11-a5db-c8dfb85cd742 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:20.687776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e488fe34-f306-4b11-a5db-c8dfb85cd742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:20.724851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:20.801849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 707/1408] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:21.093835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdd44a-26e9-47b2-9b23-05d4ddb78b46 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:21.095929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-70bdd44a-26e9-47b2-9b23-05d4ddb78b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:21.137401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:21.265649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdd44a-26e9-47b2-9b23-05d4ddb78b46 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:21.386273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:21.397844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46147c-81f1-4857-9683-77fa648952f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:21.398460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46147c-81f1-4857-9683-77fa648952f3 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:21.398736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46147c-81f1-4857-9683-77fa648952f3 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:21.399005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46147c-81f1-4857-9683-77fa648952f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:21.400778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6f46147c-81f1-4857-9683-77fa648952f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:21.500952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46147c-81f1-4857-9683-77fa648952f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:21.502626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46147c-81f1-4857-9683-77fa648952f3 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:21.521351 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 702/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:30.800777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=30 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:31.518862 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=48 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:35.206955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3e64-c6a2-4028-b509-40d3339253c3 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:35.207555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3e64-c6a2-4028-b509-40d3339253c3 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:35.207555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3e64-c6a2-4028-b509-40d3339253c3 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:35.208848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 708/1410] 10.222.0.46 () {54 vars in 918 bytes} [Mon Oct 3 14:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:35.214047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084abde-20d0-410d-ae81-15909d3999a0 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:35.214438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084abde-20d0-410d-ae81-15909d3999a0 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:35.214594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084abde-20d0-410d-ae81-15909d3999a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:35.292878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-0084abde-20d0-410d-ae81-15909d3999a0 None None] MFA Rules not processed for user `ebc01c87551044818e1ccbb46b8e7340`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:35.292878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0084abde-20d0-410d-ae81-15909d3999a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:35.399228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 703/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Oct 3 14:45:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:35.414726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e533ad-dc4c-4194-a7b7-86403172d625 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:35.415014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e533ad-dc4c-4194-a7b7-86403172d625 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:35.415274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e533ad-dc4c-4194-a7b7-86403172d625 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:35.416630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 709/1412] 10.222.0.46 () {54 vars in 918 bytes} [Mon Oct 3 14:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:35.421350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6baaa-ab35-4228-97bb-ec964bf67878 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:35.421673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6baaa-ab35-4228-97bb-ec964bf67878 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:35.421956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6baaa-ab35-4228-97bb-ec964bf67878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:35.494116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:35.497167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-65f6baaa-ab35-4228-97bb-ec964bf67878 None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:35.499013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-65f6baaa-ab35-4228-97bb-ec964bf67878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:35.611414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 704/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Oct 3 14:45:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:38.181371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71992ff8-5028-409a-bb18-1212614f1e44 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:38.181964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71992ff8-5028-409a-bb18-1212614f1e44 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:38.181964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71992ff8-5028-409a-bb18-1212614f1e44 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:38.183306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:38.189288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642924b7-ca59-4ce8-86b1-c4cbdac96068 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:38.189758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642924b7-ca59-4ce8-86b1-c4cbdac96068 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:38.189918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642924b7-ca59-4ce8-86b1-c4cbdac96068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:38.254341 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-642924b7-ca59-4ce8-86b1-c4cbdac96068 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:38.255518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-642924b7-ca59-4ce8-86b1-c4cbdac96068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:38.311132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:38.352265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:38.638712 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71992ff8-5028-409a-bb18-1212614f1e44 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:38.640462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-71992ff8-5028-409a-bb18-1212614f1e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:38.784039 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71992ff8-5028-409a-bb18-1212614f1e44 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:38.855536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:38.895398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e6513-7bcb-4f33-8d35-e8aa60d487e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:38.895934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e6513-7bcb-4f33-8d35-e8aa60d487e1 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:38.896210 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e6513-7bcb-4f33-8d35-e8aa60d487e1 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:38.896478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e6513-7bcb-4f33-8d35-e8aa60d487e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:38.898083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-073e6513-7bcb-4f33-8d35-e8aa60d487e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:38.985817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e6513-7bcb-4f33-8d35-e8aa60d487e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:38.987429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e6513-7bcb-4f33-8d35-e8aa60d487e1 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:39.005480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 711/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:40.129413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab4cda-b5c9-451c-965d-07d4aac8f161 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:40.129975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab4cda-b5c9-451c-965d-07d4aac8f161 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:40.129975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab4cda-b5c9-451c-965d-07d4aac8f161 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:40.131367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:40.137421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e760d-b1a3-4ed2-8131-876fd3e06335 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:40.137905 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e760d-b1a3-4ed2-8131-876fd3e06335 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:40.138022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e760d-b1a3-4ed2-8131-876fd3e06335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:40.213357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ca6e760d-b1a3-4ed2-8131-876fd3e06335 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:40.214588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ca6e760d-b1a3-4ed2-8131-876fd3e06335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:40.241292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:40.311516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:40.462320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab4cda-b5c9-451c-965d-07d4aac8f161 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:40.463843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b7ab4cda-b5c9-451c-965d-07d4aac8f161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:40.606748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab4cda-b5c9-451c-965d-07d4aac8f161 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:40.647200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:40.719829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0da8c4-ba64-4876-9cbf-8c961f4f59ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:40.720345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0da8c4-ba64-4876-9cbf-8c961f4f59ae service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:40.720626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0da8c4-ba64-4876-9cbf-8c961f4f59ae service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:40.720896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0da8c4-ba64-4876-9cbf-8c961f4f59ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:40.722656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0f0da8c4-ba64-4876-9cbf-8c961f4f59ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:40.810433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0da8c4-ba64-4876-9cbf-8c961f4f59ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:40.812013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0da8c4-ba64-4876-9cbf-8c961f4f59ae service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:40.829425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 707/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:41.959843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25ee0c-64b6-4070-bc0f-a8004567bb2f None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:41.960373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25ee0c-64b6-4070-bc0f-a8004567bb2f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:41.960524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25ee0c-64b6-4070-bc0f-a8004567bb2f None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:41.961970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:41.968275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e014e3-e38a-4a00-91c7-896b5a4935c9 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:41.968714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e014e3-e38a-4a00-91c7-896b5a4935c9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:41.968845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e014e3-e38a-4a00-91c7-896b5a4935c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:42.021946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:42.045611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-42e014e3-e38a-4a00-91c7-896b5a4935c9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:42.046892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-42e014e3-e38a-4a00-91c7-896b5a4935c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:42.141039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:42.291350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25ee0c-64b6-4070-bc0f-a8004567bb2f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:42.293006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3f25ee0c-64b6-4070-bc0f-a8004567bb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:42.434180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25ee0c-64b6-4070-bc0f-a8004567bb2f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:42.446484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:42.544445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d1be6-6b2a-4de4-ad45-f388cc391c08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:42.545040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1be6-6b2a-4de4-ad45-f388cc391c08 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:42.545314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1be6-6b2a-4de4-ad45-f388cc391c08 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:42.545609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1be6-6b2a-4de4-ad45-f388cc391c08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:42.547711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c47d1be6-6b2a-4de4-ad45-f388cc391c08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:42.640740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d1be6-6b2a-4de4-ad45-f388cc391c08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:42.642378 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d1be6-6b2a-4de4-ad45-f388cc391c08 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:42.659771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 714/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:43.763382 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f8c49-6665-4334-9b1e-7363ac159666 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:43.763925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f8c49-6665-4334-9b1e-7363ac159666 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:43.763925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f8c49-6665-4334-9b1e-7363ac159666 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:43.765275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:43.771944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fb250-c59a-469b-9d01-e1e8bd92e0c0 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:43.772395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fb250-c59a-469b-9d01-e1e8bd92e0c0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:43.772493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fb250-c59a-469b-9d01-e1e8bd92e0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:43.792813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:43.849992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-819fb250-c59a-469b-9d01-e1e8bd92e0c0 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:43.851207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-819fb250-c59a-469b-9d01-e1e8bd92e0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:43.946247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:44.096195 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f8c49-6665-4334-9b1e-7363ac159666 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:44.097710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2b4f8c49-6665-4334-9b1e-7363ac159666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:44.204880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:44.240898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f8c49-6665-4334-9b1e-7363ac159666 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:44.356791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fc10d-abd8-4915-92bb-6dfa9e1091b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:44.357320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fc10d-abd8-4915-92bb-6dfa9e1091b9 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:44.357624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fc10d-abd8-4915-92bb-6dfa9e1091b9 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:44.357897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fc10d-abd8-4915-92bb-6dfa9e1091b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:44.359520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f41fc10d-abd8-4915-92bb-6dfa9e1091b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:44.433519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:44.447451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fc10d-abd8-4915-92bb-6dfa9e1091b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:44.449062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fc10d-abd8-4915-92bb-6dfa9e1091b9 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:44.467210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 710/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:45.583727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90bdc7-fdb8-48d5-873e-f8ad21142e01 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:45.584319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90bdc7-fdb8-48d5-873e-f8ad21142e01 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:45.584319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90bdc7-fdb8-48d5-873e-f8ad21142e01 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:45.585563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:45.591963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e34e01-512f-4037-9c60-a71e8584c525 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:45.592385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e34e01-512f-4037-9c60-a71e8584c525 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:45.592576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e34e01-512f-4037-9c60-a71e8584c525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:45.667743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e9e34e01-512f-4037-9c60-a71e8584c525 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:45.668954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e9e34e01-512f-4037-9c60-a71e8584c525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:45.764026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:45.915471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90bdc7-fdb8-48d5-873e-f8ad21142e01 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:45.917024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ac90bdc7-fdb8-48d5-873e-f8ad21142e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:45.999947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:46.060956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90bdc7-fdb8-48d5-873e-f8ad21142e01 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:46.170710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc6956-957e-4b34-b901-683aa77ae424 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:46.171308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc6956-957e-4b34-b901-683aa77ae424 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:46.171583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc6956-957e-4b34-b901-683aa77ae424 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:46.171873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc6956-957e-4b34-b901-683aa77ae424 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:46.173441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8bdc6956-957e-4b34-b901-683aa77ae424 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:46.221018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:46.261491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc6956-957e-4b34-b901-683aa77ae424 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:46.263127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc6956-957e-4b34-b901-683aa77ae424 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:46.282561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 717/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:47.332040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec48fcb-fcff-47a3-903d-31521ad00144 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:47.332556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec48fcb-fcff-47a3-903d-31521ad00144 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:47.332659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec48fcb-fcff-47a3-903d-31521ad00144 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:47.333989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:47.340198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5e42-c9ef-4628-af06-9a0031d2b63f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:47.340639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5e42-c9ef-4628-af06-9a0031d2b63f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:47.340747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5e42-c9ef-4628-af06-9a0031d2b63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:47.534241 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-852d5e42-c9ef-4628-af06-9a0031d2b63f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:47.535513 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-852d5e42-c9ef-4628-af06-9a0031d2b63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:47.629266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:47.777279 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec48fcb-fcff-47a3-903d-31521ad00144 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:47.778859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9ec48fcb-fcff-47a3-903d-31521ad00144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:47.827134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:47.925019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec48fcb-fcff-47a3-903d-31521ad00144 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:48.035082 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbfbb7-e09e-49bf-ba43-f85eb8261013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:48.035689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfbb7-e09e-49bf-ba43-f85eb8261013 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:48.035967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfbb7-e09e-49bf-ba43-f85eb8261013 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:48.036230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfbb7-e09e-49bf-ba43-f85eb8261013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:48.037896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8dfbfbb7-e09e-49bf-ba43-f85eb8261013 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:48.044953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:48.126586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbfbb7-e09e-49bf-ba43-f85eb8261013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:48.128178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbfbb7-e09e-49bf-ba43-f85eb8261013 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:48.146181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 713/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:49.196595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd73b8-99ef-4054-8420-258d73e4a383 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:49.197137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd73b8-99ef-4054-8420-258d73e4a383 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:49.197137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd73b8-99ef-4054-8420-258d73e4a383 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:49.198493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:49.204777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd10e0-4ef7-47f6-885e-c5c33efe3aaa None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:49.205231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd10e0-4ef7-47f6-885e-c5c33efe3aaa None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:49.205339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd10e0-4ef7-47f6-885e-c5c33efe3aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:49.280162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-81cd10e0-4ef7-47f6-885e-c5c33efe3aaa None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:49.281414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-81cd10e0-4ef7-47f6-885e-c5c33efe3aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:49.347860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:49.376496 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:49.526140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd73b8-99ef-4054-8420-258d73e4a383 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:49.527567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-0ecd73b8-99ef-4054-8420-258d73e4a383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:49.546287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:49.671225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd73b8-99ef-4054-8420-258d73e4a383 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:49.754873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:49.781440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc4c9b-8c2b-4596-b56a-aa3457a5be55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:49.782008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4c9b-8c2b-4596-b56a-aa3457a5be55 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:49.782285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4c9b-8c2b-4596-b56a-aa3457a5be55 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:49.782554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4c9b-8c2b-4596-b56a-aa3457a5be55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:49.784132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c4dc4c9b-8c2b-4596-b56a-aa3457a5be55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:49.870829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc4c9b-8c2b-4596-b56a-aa3457a5be55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:49.872414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc4c9b-8c2b-4596-b56a-aa3457a5be55 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:49.891027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 720/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:50.933954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106c120-4cd6-4148-aabf-51c9bd082c4d None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:50.934477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106c120-4cd6-4148-aabf-51c9bd082c4d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:50.934592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106c120-4cd6-4148-aabf-51c9bd082c4d None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:50.935871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:50.942560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2833571-c33a-43c3-a446-a8ce54722dc0 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:50.943008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2833571-c33a-43c3-a446-a8ce54722dc0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:50.943111 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2833571-c33a-43c3-a446-a8ce54722dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:51.019438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b2833571-c33a-43c3-a446-a8ce54722dc0 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:51.020642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b2833571-c33a-43c3-a446-a8ce54722dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:51.061999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:51.116411 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:51.264634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106c120-4cd6-4148-aabf-51c9bd082c4d None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:51.266144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3106c120-4cd6-4148-aabf-51c9bd082c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:51.411063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106c120-4cd6-4148-aabf-51c9bd082c4d None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:51.463897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:51.522533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcde98-490a-4bec-8ade-bc0753e7c986 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:51.523072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcde98-490a-4bec-8ade-bc0753e7c986 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:51.523353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcde98-490a-4bec-8ade-bc0753e7c986 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:51.523647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcde98-490a-4bec-8ade-bc0753e7c986 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:51.525253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-11dcde98-490a-4bec-8ade-bc0753e7c986 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:51.614851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcde98-490a-4bec-8ade-bc0753e7c986 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:51.616442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcde98-490a-4bec-8ade-bc0753e7c986 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:51.634184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 716/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:52.677230 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b454f5-09fb-47bb-b6a5-a4622bc4d173 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:52.677797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b454f5-09fb-47bb-b6a5-a4622bc4d173 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:52.677797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b454f5-09fb-47bb-b6a5-a4622bc4d173 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:52.679146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:52.685219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e5fe-a71c-44a3-a665-a9c76484bcf9 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:52.685687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e5fe-a71c-44a3-a665-a9c76484bcf9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:52.685790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e5fe-a71c-44a3-a665-a9c76484bcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:52.758570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-49b9e5fe-a71c-44a3-a665-a9c76484bcf9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:52.759779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-49b9e5fe-a71c-44a3-a665-a9c76484bcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:52.761781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:52.852957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:53.003647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b454f5-09fb-47bb-b6a5-a4622bc4d173 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:53.005093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c0b454f5-09fb-47bb-b6a5-a4622bc4d173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:53.150855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b454f5-09fb-47bb-b6a5-a4622bc4d173 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:53.177366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:53.263205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c618b8-7072-4434-a2ba-00b58addd6fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:53.263725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c618b8-7072-4434-a2ba-00b58addd6fd service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:53.263990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c618b8-7072-4434-a2ba-00b58addd6fd service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:53.264249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c618b8-7072-4434-a2ba-00b58addd6fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:53.266351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-42c618b8-7072-4434-a2ba-00b58addd6fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:53.353912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c618b8-7072-4434-a2ba-00b58addd6fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:53.355493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c618b8-7072-4434-a2ba-00b58addd6fd service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:53.372703 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 723/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:55.425014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9a0e-c6a8-492d-bb13-d286cb4152b0 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:55.425581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9a0e-c6a8-492d-bb13-d286cb4152b0 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:55.425581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9a0e-c6a8-492d-bb13-d286cb4152b0 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:55.426907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:55.432268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640fe8c-8164-4ccf-a8c6-c3f2e3a3499f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:55.432724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640fe8c-8164-4ccf-a8c6-c3f2e3a3499f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:55.432879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640fe8c-8164-4ccf-a8c6-c3f2e3a3499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:55.473515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:55.498156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d640fe8c-8164-4ccf-a8c6-c3f2e3a3499f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:55.499341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d640fe8c-8164-4ccf-a8c6-c3f2e3a3499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:55.599215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:55.909203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc9a0e-c6a8-492d-bb13-d286cb4152b0 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:55.910846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c7dc9a0e-c6a8-492d-bb13-d286cb4152b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:56.012257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc9a0e-c6a8-492d-bb13-d286cb4152b0 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:56.043071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:56.086519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cd86b-e34a-43cb-bd36-4a94f8c3f8c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:56.087125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cd86b-e34a-43cb-bd36-4a94f8c3f8c0 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:56.087404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cd86b-e34a-43cb-bd36-4a94f8c3f8c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:56.087674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cd86b-e34a-43cb-bd36-4a94f8c3f8c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:56.089274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f32cd86b-e34a-43cb-bd36-4a94f8c3f8c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:56.198253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cd86b-e34a-43cb-bd36-4a94f8c3f8c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:56.199704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cd86b-e34a-43cb-bd36-4a94f8c3f8c0 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:56.217655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 719/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:45:58.505031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41619836-6a82-4154-9b1d-34a1a096caa3 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:58.505576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41619836-6a82-4154-9b1d-34a1a096caa3 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:58.505576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41619836-6a82-4154-9b1d-34a1a096caa3 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:58.506849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:45:58.511980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102390-1db6-423f-aee8-1d5e8e914e12 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:58.512371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102390-1db6-423f-aee8-1d5e8e914e12 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:58.512531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102390-1db6-423f-aee8-1d5e8e914e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:58.576555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-65102390-1db6-423f-aee8-1d5e8e914e12 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:45:58.577824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-65102390-1db6-423f-aee8-1d5e8e914e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:58.590181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:58.673842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:45:58.958432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41619836-6a82-4154-9b1d-34a1a096caa3 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:45:58.959949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-41619836-6a82-4154-9b1d-34a1a096caa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:59.060422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41619836-6a82-4154-9b1d-34a1a096caa3 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:45:59.073660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:45:59.134916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5cc6c-c09b-4a11-9cc3-787178c9cc06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:45:59.135514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5cc6c-c09b-4a11-9cc3-787178c9cc06 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:45:59.135796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5cc6c-c09b-4a11-9cc3-787178c9cc06 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:45:59.136070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5cc6c-c09b-4a11-9cc3-787178c9cc06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:45:59.137691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6ce5cc6c-c09b-4a11-9cc3-787178c9cc06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:45:59.244861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5cc6c-c09b-4a11-9cc3-787178c9cc06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:45:59.246299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5cc6c-c09b-4a11-9cc3-787178c9cc06 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:45:59.264217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 726/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:45:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:01.711552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b3950-6664-41c3-a869-610540436bab None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:01.712080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b3950-6664-41c3-a869-610540436bab None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:01.712080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b3950-6664-41c3-a869-610540436bab None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:01.713461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:01.719304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05cd7e-0ea1-454a-a953-8963323857bf None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:01.719730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05cd7e-0ea1-454a-a953-8963323857bf None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:01.719845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05cd7e-0ea1-454a-a953-8963323857bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:01.766452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:01.784539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-8c05cd7e-0ea1-454a-a953-8963323857bf None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:01.785746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8c05cd7e-0ea1-454a-a953-8963323857bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:01.879028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:02.160144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b3950-6664-41c3-a869-610540436bab None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:02.161738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c51b3950-6664-41c3-a869-610540436bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:02.261381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b3950-6664-41c3-a869-610540436bab None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:02.301316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:02.335513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418b6c5-8272-4c62-ae7d-d7f57572e3da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:02.336114 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418b6c5-8272-4c62-ae7d-d7f57572e3da service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:02.336392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418b6c5-8272-4c62-ae7d-d7f57572e3da service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:02.336663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418b6c5-8272-4c62-ae7d-d7f57572e3da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:02.338301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e418b6c5-8272-4c62-ae7d-d7f57572e3da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:02.443815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418b6c5-8272-4c62-ae7d-d7f57572e3da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:02.445208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418b6c5-8272-4c62-ae7d-d7f57572e3da service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:02.463211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 722/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:04.709007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4232e5e-ec06-4713-add7-714e1ce7f9cd None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:04.709591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4232e5e-ec06-4713-add7-714e1ce7f9cd None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:04.709591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4232e5e-ec06-4713-add7-714e1ce7f9cd None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:04.710909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:04.716146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6c08-9e40-4808-afed-8083a57eef22 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:04.716571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6c08-9e40-4808-afed-8083a57eef22 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:04.716686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6c08-9e40-4808-afed-8083a57eef22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:04.782047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-41ea6c08-9e40-4808-afed-8083a57eef22 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:04.783254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-41ea6c08-9e40-4808-afed-8083a57eef22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:04.803834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:04.879005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:05.163582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4232e5e-ec06-4713-add7-714e1ce7f9cd None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:05.165139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e4232e5e-ec06-4713-add7-714e1ce7f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:05.265513 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4232e5e-ec06-4713-add7-714e1ce7f9cd None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:05.286294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:05.341785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78290b4-86c4-4e9e-8dd3-50d8bd83b55e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:05.342388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78290b4-86c4-4e9e-8dd3-50d8bd83b55e service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:05.342667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78290b4-86c4-4e9e-8dd3-50d8bd83b55e service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:05.342941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78290b4-86c4-4e9e-8dd3-50d8bd83b55e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:05.344543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a78290b4-86c4-4e9e-8dd3-50d8bd83b55e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:05.452655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78290b4-86c4-4e9e-8dd3-50d8bd83b55e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:05.454067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78290b4-86c4-4e9e-8dd3-50d8bd83b55e service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:05.471631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 729/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:07.765526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d9a71-6fe0-477f-ba38-4d508f39985a None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:07.766152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d9a71-6fe0-477f-ba38-4d508f39985a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:07.766152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d9a71-6fe0-477f-ba38-4d508f39985a None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:07.767424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:07.772790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6f4e-ffe6-4e01-a800-f302d71d25ec None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:07.773197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6f4e-ffe6-4e01-a800-f302d71d25ec None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:07.773349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6f4e-ffe6-4e01-a800-f302d71d25ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:07.834843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:07.838021 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d89a6f4e-ffe6-4e01-a800-f302d71d25ec None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:07.839793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d89a6f4e-ffe6-4e01-a800-f302d71d25ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:07.934135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:08.216159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d9a71-6fe0-477f-ba38-4d508f39985a None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:08.217789 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0d1d9a71-6fe0-477f-ba38-4d508f39985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:08.320639 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d9a71-6fe0-477f-ba38-4d508f39985a None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:08.369237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:08.392936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a3eea-7742-4550-aabf-56263a5ba2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:08.393561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3eea-7742-4550-aabf-56263a5ba2fe service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:08.393844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3eea-7742-4550-aabf-56263a5ba2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:08.394113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a3eea-7742-4550-aabf-56263a5ba2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:08.395693 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f81a3eea-7742-4550-aabf-56263a5ba2fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:08.500541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a3eea-7742-4550-aabf-56263a5ba2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:08.501943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a3eea-7742-4550-aabf-56263a5ba2fe service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:08.519265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 725/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:10.703910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff940b-f0f5-466c-a556-fa99f67d8ec0 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:10.704472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff940b-f0f5-466c-a556-fa99f67d8ec0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:10.704472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff940b-f0f5-466c-a556-fa99f67d8ec0 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:10.705868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:10.711288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac028-51e6-4170-846d-01fd2f6b71d2 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:10.711688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac028-51e6-4170-846d-01fd2f6b71d2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:10.711840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac028-51e6-4170-846d-01fd2f6b71d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:10.778092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e0fac028-51e6-4170-846d-01fd2f6b71d2 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:10.779310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e0fac028-51e6-4170-846d-01fd2f6b71d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:10.807461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:10.874993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:11.156567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff940b-f0f5-466c-a556-fa99f67d8ec0 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:11.158134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-59ff940b-f0f5-466c-a556-fa99f67d8ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:11.263386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff940b-f0f5-466c-a556-fa99f67d8ec0 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:11.292067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:11.336204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bc293-ffca-4f2d-a175-ca4a84ae0822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:11.336800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bc293-ffca-4f2d-a175-ca4a84ae0822 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:11.337072 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bc293-ffca-4f2d-a175-ca4a84ae0822 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:11.337338 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bc293-ffca-4f2d-a175-ca4a84ae0822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:11.338931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1d4bc293-ffca-4f2d-a175-ca4a84ae0822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:11.444017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bc293-ffca-4f2d-a175-ca4a84ae0822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:11.445426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bc293-ffca-4f2d-a175-ca4a84ae0822 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:11.462977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 732/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:46:13.660710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87057f30-253d-4632-b072-839f10cff011 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:13.661229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87057f30-253d-4632-b072-839f10cff011 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:13.661229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87057f30-253d-4632-b072-839f10cff011 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:13.662619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:13.667964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b63d1-30d4-4e1c-b8b6-7348b87c40d6 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:13.668366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b63d1-30d4-4e1c-b8b6-7348b87c40d6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:13.668511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b63d1-30d4-4e1c-b8b6-7348b87c40d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:13.733604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-129b63d1-30d4-4e1c-b8b6-7348b87c40d6 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:13.734793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-129b63d1-30d4-4e1c-b8b6-7348b87c40d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:13.742796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:13.831249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:14.113933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87057f30-253d-4632-b072-839f10cff011 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:14.115503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-87057f30-253d-4632-b072-839f10cff011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:14.216036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87057f30-253d-4632-b072-839f10cff011 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:14.270687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:14.289820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c91a6-6cc2-430e-aa7f-afb040f79cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:14.290432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c91a6-6cc2-430e-aa7f-afb040f79cff service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:14.290711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c91a6-6cc2-430e-aa7f-afb040f79cff service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:14.290983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c91a6-6cc2-430e-aa7f-afb040f79cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:14.292598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9d5c91a6-6cc2-430e-aa7f-afb040f79cff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:14.398936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c91a6-6cc2-430e-aa7f-afb040f79cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:14.400303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c91a6-6cc2-430e-aa7f-afb040f79cff service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:14.417776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 728/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:16.618941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8fe36-02a2-416e-a8e9-44633687de8b None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:16.619492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8fe36-02a2-416e-a8e9-44633687de8b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:16.619492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8fe36-02a2-416e-a8e9-44633687de8b None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:16.620782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:16.626594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5468a-3224-4e44-9ea2-8035691bf31a None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:16.627016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5468a-3224-4e44-9ea2-8035691bf31a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:16.627146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5468a-3224-4e44-9ea2-8035691bf31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:16.694225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6ca5468a-3224-4e44-9ea2-8035691bf31a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:16.695435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6ca5468a-3224-4e44-9ea2-8035691bf31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:16.733391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:16.790641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:17.073660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8fe36-02a2-416e-a8e9-44633687de8b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:17.075222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-02c8fe36-02a2-416e-a8e9-44633687de8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:17.176462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8fe36-02a2-416e-a8e9-44633687de8b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:17.213297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:17.250580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e3902-9ddc-4b9c-8257-63b7155d9727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:17.251175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e3902-9ddc-4b9c-8257-63b7155d9727 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:17.251450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e3902-9ddc-4b9c-8257-63b7155d9727 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:17.251714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e3902-9ddc-4b9c-8257-63b7155d9727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:17.253293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-699e3902-9ddc-4b9c-8257-63b7155d9727 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:17.361228 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e3902-9ddc-4b9c-8257-63b7155d9727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:17.362618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e3902-9ddc-4b9c-8257-63b7155d9727 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:17.380425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 735/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:19.543342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3346c2-b6ac-4e7d-94a2-569b84975884 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:19.543872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3346c2-b6ac-4e7d-94a2-569b84975884 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:19.543872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3346c2-b6ac-4e7d-94a2-569b84975884 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:19.545292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:19.550732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd1086-4296-4c76-b1d2-780ccde0f146 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:19.551151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd1086-4296-4c76-b1d2-780ccde0f146 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:19.551300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd1086-4296-4c76-b1d2-780ccde0f146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:19.616112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ddfd1086-4296-4c76-b1d2-780ccde0f146 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:19.617310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ddfd1086-4296-4c76-b1d2-780ccde0f146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:19.635243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:19.712533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:19.991721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3346c2-b6ac-4e7d-94a2-569b84975884 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:19.993258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1f3346c2-b6ac-4e7d-94a2-569b84975884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:20.094038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3346c2-b6ac-4e7d-94a2-569b84975884 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:20.159302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:20.166114 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989a6ac-092f-4ca7-b3d4-794d8c769ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:20.166711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989a6ac-092f-4ca7-b3d4-794d8c769ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:20.166985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989a6ac-092f-4ca7-b3d4-794d8c769ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:20.167255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989a6ac-092f-4ca7-b3d4-794d8c769ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:20.168831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0989a6ac-092f-4ca7-b3d4-794d8c769ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:20.276329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989a6ac-092f-4ca7-b3d4-794d8c769ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:20.277893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989a6ac-092f-4ca7-b3d4-794d8c769ef4 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:20.297091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 731/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:22.440845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce36c9e-9a97-4a8c-a552-b7b055dc6621 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:22.441370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce36c9e-9a97-4a8c-a552-b7b055dc6621 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:22.441370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce36c9e-9a97-4a8c-a552-b7b055dc6621 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:22.442766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:22.448079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d91a3-85fb-465e-ab20-171136a11715 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:22.448513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d91a3-85fb-465e-ab20-171136a11715 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:22.448642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d91a3-85fb-465e-ab20-171136a11715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:22.512791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-304d91a3-85fb-465e-ab20-171136a11715 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:22.514039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-304d91a3-85fb-465e-ab20-171136a11715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:22.561910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:22.725077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:23.004777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce36c9e-9a97-4a8c-a552-b7b055dc6621 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:23.006451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6ce36c9e-9a97-4a8c-a552-b7b055dc6621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:23.111795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce36c9e-9a97-4a8c-a552-b7b055dc6621 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:23.159331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:23.184680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87732c69-e64d-48d2-9f0d-79f3a439840c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:23.185282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732c69-e64d-48d2-9f0d-79f3a439840c service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:23.185589 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732c69-e64d-48d2-9f0d-79f3a439840c service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:23.185862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732c69-e64d-48d2-9f0d-79f3a439840c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:23.187486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-87732c69-e64d-48d2-9f0d-79f3a439840c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:23.296676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87732c69-e64d-48d2-9f0d-79f3a439840c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:23.298157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87732c69-e64d-48d2-9f0d-79f3a439840c service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:23.316531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 738/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:32.723580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=22 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:33.314324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=66 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:43.225368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d0aa4-6fbe-4702-9951-443f4398bced None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:43.225938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d0aa4-6fbe-4702-9951-443f4398bced None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:43.225938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d0aa4-6fbe-4702-9951-443f4398bced None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:43.227561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 733/1471] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:43.233600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817f39c-9695-4cd0-8a68-cb9f82240ed0 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:43.234011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817f39c-9695-4cd0-8a68-cb9f82240ed0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:43.234141 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817f39c-9695-4cd0-8a68-cb9f82240ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:43.299464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-2817f39c-9695-4cd0-8a68-cb9f82240ed0 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:43.300669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2817f39c-9695-4cd0-8a68-cb9f82240ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:43.394662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 739/1472] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:46:43.693444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12776c3-6442-4b0b-9b98-4ae3eafe37be None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:43.693760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12776c3-6442-4b0b-9b98-4ae3eafe37be None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:43.694034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12776c3-6442-4b0b-9b98-4ae3eafe37be None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:43.695345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 734/1473] 10.222.0.46 () {54 vars in 962 bytes} [Mon Oct 3 14:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:43.700901 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6e421-a415-4ee0-8f69-13356d283dcc None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:43.701182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6e421-a415-4ee0-8f69-13356d283dcc None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:43.701450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6e421-a415-4ee0-8f69-13356d283dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:43.775777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-76b6e421-a415-4ee0-8f69-13356d283dcc None None] MFA Rules not processed for user `dd04df68f9d84fc4bb981c8b69cc771a`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:43.776963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-76b6e421-a415-4ee0-8f69-13356d283dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:43.796707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:43.850462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 740/1474] 10.222.0.46 () {58 vars in 1059 bytes} [Mon Oct 3 14:46:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:43.858359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12776c3-6442-4b0b-9b98-4ae3eafe37be None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:43.859870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a12776c3-6442-4b0b-9b98-4ae3eafe37be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:43.961917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12776c3-6442-4b0b-9b98-4ae3eafe37be None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:44.033933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f40d81-9c12-4e76-8dca-8d6b4038e8aa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd04df68f9d84fc4bb981c8b69cc771a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:44.034542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f40d81-9c12-4e76-8dca-8d6b4038e8aa service glance] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:44.034823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f40d81-9c12-4e76-8dca-8d6b4038e8aa service glance] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:44.035096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f40d81-9c12-4e76-8dca-8d6b4038e8aa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:44.036700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-39f40d81-9c12-4e76-8dca-8d6b4038e8aa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:44.090542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:44.144571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f40d81-9c12-4e76-8dca-8d6b4038e8aa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:44.146040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f40d81-9c12-4e76-8dca-8d6b4038e8aa service glance] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:44.163358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 735/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:46:46.404686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0de2d0-2406-4cd1-a833-03373364bd5a None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:46.405215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0de2d0-2406-4cd1-a833-03373364bd5a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:46.405215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0de2d0-2406-4cd1-a833-03373364bd5a None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:46.406606 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 741/1476] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:46.411814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c1b92-d933-4943-a38b-f8dd88e8492c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:46.412249 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c1b92-d933-4943-a38b-f8dd88e8492c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:46.412391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c1b92-d933-4943-a38b-f8dd88e8492c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:46.479835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-991c1b92-d933-4943-a38b-f8dd88e8492c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:46.481035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-991c1b92-d933-4943-a38b-f8dd88e8492c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:46.576388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 736/1477] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:46.850648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0de2d0-2406-4cd1-a833-03373364bd5a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:46.852364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4f0de2d0-2406-4cd1-a833-03373364bd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:46.978556 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0de2d0-2406-4cd1-a833-03373364bd5a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:47.027146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:47.071272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef25c1-5a83-49fb-a993-b9a07476cc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:47.071873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef25c1-5a83-49fb-a993-b9a07476cc36 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:47.072147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef25c1-5a83-49fb-a993-b9a07476cc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:47.072413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef25c1-5a83-49fb-a993-b9a07476cc36 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:47.073042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef25c1-5a83-49fb-a993-b9a07476cc36 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:47.074809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WARNING py.warnings [None req-33ef25c1-5a83-49fb-a993-b9a07476cc36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:46:47.074809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: warnings.warn(msg) Oct 03 14:46:47.074809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: Oct 03 14:46:47.075939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef25c1-5a83-49fb-a993-b9a07476cc36 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:47.094389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 742/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 3 14:46:46 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:46:49.248756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399cb13-7e8b-47f8-bf3f-9db834b74303 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:49.249325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399cb13-7e8b-47f8-bf3f-9db834b74303 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:49.249325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399cb13-7e8b-47f8-bf3f-9db834b74303 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:49.250718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 737/1479] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:49.255983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824f287-101a-4188-af31-13a924775360 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:49.256418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824f287-101a-4188-af31-13a924775360 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:49.256520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824f287-101a-4188-af31-13a924775360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:49.321525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e824f287-101a-4188-af31-13a924775360 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:49.322720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e824f287-101a-4188-af31-13a924775360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:49.415869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 743/1480] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:49.708209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399cb13-7e8b-47f8-bf3f-9db834b74303 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:49.709760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9399cb13-7e8b-47f8-bf3f-9db834b74303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:49.719753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:49.856634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399cb13-7e8b-47f8-bf3f-9db834b74303 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:49.934159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:49.966664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93c869-7627-4d90-ad5f-01786bb76113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:49.967186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93c869-7627-4d90-ad5f-01786bb76113 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:49.967482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93c869-7627-4d90-ad5f-01786bb76113 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:49.967748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93c869-7627-4d90-ad5f-01786bb76113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:49.969328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0e93c869-7627-4d90-ad5f-01786bb76113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:50.056088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93c869-7627-4d90-ad5f-01786bb76113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:50.057718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93c869-7627-4d90-ad5f-01786bb76113 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:50.075839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 738/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:52.375395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1e8a0-bd0f-42e3-afe6-da21a3906af0 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:52.375910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1e8a0-bd0f-42e3-afe6-da21a3906af0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:52.375910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1e8a0-bd0f-42e3-afe6-da21a3906af0 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:52.377260 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 744/1482] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:52.382621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ed900-29ed-41dc-b5fd-67c735514898 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:52.383049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ed900-29ed-41dc-b5fd-67c735514898 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:52.383170 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ed900-29ed-41dc-b5fd-67c735514898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:52.448406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-100ed900-29ed-41dc-b5fd-67c735514898 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:52.449685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-100ed900-29ed-41dc-b5fd-67c735514898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:52.496186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:52.547314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 739/1483] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:52.833924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1e8a0-bd0f-42e3-afe6-da21a3906af0 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:52.835448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f8f1e8a0-bd0f-42e3-afe6-da21a3906af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:52.859813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:52.937178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1e8a0-bd0f-42e3-afe6-da21a3906af0 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:53.009826 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f32a6-5c31-44b2-8b15-8d915d7576f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:53.010430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f32a6-5c31-44b2-8b15-8d915d7576f9 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:53.010706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f32a6-5c31-44b2-8b15-8d915d7576f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:53.010973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f32a6-5c31-44b2-8b15-8d915d7576f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:53.013070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a25f32a6-5c31-44b2-8b15-8d915d7576f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:53.088751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:53.123910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f32a6-5c31-44b2-8b15-8d915d7576f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:53.125356 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f32a6-5c31-44b2-8b15-8d915d7576f9 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:53.143255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 745/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:55.304444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff13f7-cf5a-49d6-b5d7-83eb61ff01a2 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:55.304933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff13f7-cf5a-49d6-b5d7-83eb61ff01a2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:55.305057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff13f7-cf5a-49d6-b5d7-83eb61ff01a2 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:55.306364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 740/1485] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:55.311882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a22b32-6261-4fb0-ac91-e28fc9da26b5 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:55.312347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a22b32-6261-4fb0-ac91-e28fc9da26b5 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:55.312549 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a22b32-6261-4fb0-ac91-e28fc9da26b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:55.377061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-76a22b32-6261-4fb0-ac91-e28fc9da26b5 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:55.378281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-76a22b32-6261-4fb0-ac91-e28fc9da26b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:55.471723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 746/1486] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:55.758028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff13f7-cf5a-49d6-b5d7-83eb61ff01a2 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:55.759603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-faff13f7-cf5a-49d6-b5d7-83eb61ff01a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:55.861117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff13f7-cf5a-49d6-b5d7-83eb61ff01a2 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:55.933106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e48c66-96e1-4ec7-a9b5-ae22e4e0d471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:55.934309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e48c66-96e1-4ec7-a9b5-ae22e4e0d471 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:55.934589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e48c66-96e1-4ec7-a9b5-ae22e4e0d471 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:55.934857 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e48c66-96e1-4ec7-a9b5-ae22e4e0d471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:55.936431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:55.937081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-06e48c66-96e1-4ec7-a9b5-ae22e4e0d471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:56.048325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e48c66-96e1-4ec7-a9b5-ae22e4e0d471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:56.050121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e48c66-96e1-4ec7-a9b5-ae22e4e0d471 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:56.069810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 741/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:46:58.311373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d779d8-f308-4bc4-abdb-4d24679d0bb3 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:58.311816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d779d8-f308-4bc4-abdb-4d24679d0bb3 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:58.311929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d779d8-f308-4bc4-abdb-4d24679d0bb3 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:58.313254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 747/1488] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:46:58.318598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e5647-6cb6-4b49-ba24-e7f9ac33a781 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:58.318986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e5647-6cb6-4b49-ba24-e7f9ac33a781 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:58.319161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e5647-6cb6-4b49-ba24-e7f9ac33a781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:58.383535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-871e5647-6cb6-4b49-ba24-e7f9ac33a781 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:46:58.384742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-871e5647-6cb6-4b49-ba24-e7f9ac33a781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:58.435883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:58.480403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 742/1489] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:46:58.770428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d779d8-f308-4bc4-abdb-4d24679d0bb3 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:46:58.771976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-48d779d8-f308-4bc4-abdb-4d24679d0bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:58.808429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:58.874137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d779d8-f308-4bc4-abdb-4d24679d0bb3 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:46:58.946700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844df4c-94cc-448b-8072-29d956db12ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:46:58.947297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844df4c-94cc-448b-8072-29d956db12ef service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:46:58.947571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844df4c-94cc-448b-8072-29d956db12ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:46:58.947836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844df4c-94cc-448b-8072-29d956db12ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:46:58.949412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5844df4c-94cc-448b-8072-29d956db12ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:46:59.026347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:46:59.055205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844df4c-94cc-448b-8072-29d956db12ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:46:59.056642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844df4c-94cc-448b-8072-29d956db12ef service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:46:59.075165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 748/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:46:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:47:01.416143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69a519-2342-4339-b00f-e262e1702483 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:47:01.416700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69a519-2342-4339-b00f-e262e1702483 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:47:01.416700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69a519-2342-4339-b00f-e262e1702483 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:47:01.418563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 743/1491] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 3 14:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:47:01.423788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c071987-f478-4300-98f9-e292ab7b42ef None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:47:01.424233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c071987-f478-4300-98f9-e292ab7b42ef None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:47:01.424345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c071987-f478-4300-98f9-e292ab7b42ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:47:01.489453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-9c071987-f478-4300-98f9-e292ab7b42ef None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:47:01.490686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9c071987-f478-4300-98f9-e292ab7b42ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:47:01.585264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 749/1492] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 3 14:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:47:01.882079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69a519-2342-4339-b00f-e262e1702483 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:47:01.883634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2b69a519-2342-4339-b00f-e262e1702483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:47:01.985117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69a519-2342-4339-b00f-e262e1702483 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:47:02.061852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafe341-91e7-4aad-9928-5cc3878b9ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:47:02.062463 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafe341-91e7-4aad-9928-5cc3878b9ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:47:02.062744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafe341-91e7-4aad-9928-5cc3878b9ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:47:02.063017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafe341-91e7-4aad-9928-5cc3878b9ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:47:02.064649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2dafe341-91e7-4aad-9928-5cc3878b9ad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:47:02.074526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:47:02.172214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafe341-91e7-4aad-9928-5cc3878b9ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:47:02.173667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafe341-91e7-4aad-9928-5cc3878b9ad2 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:47:02.191066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 744/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:47:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:47:11.583768 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=82 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:47:12.188981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=50 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:22.324578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200cfd1-5320-408d-8d9b-cce65e4511bf None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:22.325089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200cfd1-5320-408d-8d9b-cce65e4511bf None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:22.325192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200cfd1-5320-408d-8d9b-cce65e4511bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:22.399441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-a200cfd1-5320-408d-8d9b-cce65e4511bf None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:48:22.400634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a200cfd1-5320-408d-8d9b-cce65e4511bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:22.496857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 750/1494] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:48:22.505026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e21855-7c67-4a77-8aff-0a69c3aa294e None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:22.505549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e21855-7c67-4a77-8aff-0a69c3aa294e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:22.505663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e21855-7c67-4a77-8aff-0a69c3aa294e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:22.578465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-22e21855-7c67-4a77-8aff-0a69c3aa294e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:48:22.579589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-22e21855-7c67-4a77-8aff-0a69c3aa294e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:22.673831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 745/1495] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:48:22.678554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200cfd1-5320-408d-8d9b-cce65e4511bf None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:48:22.679789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a200cfd1-5320-408d-8d9b-cce65e4511bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:22.691047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200cfd1-5320-408d-8d9b-cce65e4511bf None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:48:22.751976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:22.785293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:48:22.785927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:22.786206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:22.786477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:22.787119 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:48:22.787421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:48:22.788987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WARNING py.warnings [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:48:22.788987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: warnings.warn(msg) Oct 03 14:48:22.788987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: Oct 03 14:48:22.789913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:48:22.801652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 751/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:48:22.809855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e21855-7c67-4a77-8aff-0a69c3aa294e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:48:22.819155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e21855-7c67-4a77-8aff-0a69c3aa294e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:48:22.880267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:22.913591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf5041-1103-4936-a924-58a886ac152c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:48:22.914187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf5041-1103-4936-a924-58a886ac152c admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:22.914461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf5041-1103-4936-a924-58a886ac152c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:22.914726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf5041-1103-4936-a924-58a886ac152c admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:22.915358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf5041-1103-4936-a924-58a886ac152c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:48:22.917091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: WARNING py.warnings [None req-8fdf5041-1103-4936-a924-58a886ac152c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:48:22.917091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: warnings.warn(msg) Oct 03 14:48:22.917091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: Oct 03 14:48:22.918317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf5041-1103-4936-a924-58a886ac152c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:48:22.972872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 746/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:48:22 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:48:22.977518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:48:22.979062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:23.011960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508dbb2-2a11-4519-9cd8-2a9b6004c84c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:48:23.103773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31852f71-6d5b-473e-9110-4c692d8d21f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:48:23.104380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31852f71-6d5b-473e-9110-4c692d8d21f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:23.104656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31852f71-6d5b-473e-9110-4c692d8d21f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:23.104925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31852f71-6d5b-473e-9110-4c692d8d21f4 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:23.105598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31852f71-6d5b-473e-9110-4c692d8d21f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:48:23.106613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WARNING py.warnings [None req-31852f71-6d5b-473e-9110-4c692d8d21f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:48:23.106613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: warnings.warn(msg) Oct 03 14:48:23.106613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: Oct 03 14:48:23.107848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31852f71-6d5b-473e-9110-4c692d8d21f4 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:48:23.149689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 752/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:48:22 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:48:23.153780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf5041-1103-4936-a924-58a886ac152c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:48:23.155305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8fdf5041-1103-4936-a924-58a886ac152c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:23.215586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf5041-1103-4936-a924-58a886ac152c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:48:23.298127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:23.309558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadefffa-51d6-4336-8405-dfbae983a94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:48:23.310164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadefffa-51d6-4336-8405-dfbae983a94f admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:23.310437 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadefffa-51d6-4336-8405-dfbae983a94f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:23.310700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadefffa-51d6-4336-8405-dfbae983a94f admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:23.311340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadefffa-51d6-4336-8405-dfbae983a94f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:48:23.312332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: WARNING py.warnings [None req-fadefffa-51d6-4336-8405-dfbae983a94f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:48:23.312332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: warnings.warn(msg) Oct 03 14:48:23.312332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: Oct 03 14:48:23.313353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadefffa-51d6-4336-8405-dfbae983a94f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:48:23.326259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 747/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:48:23.330242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31852f71-6d5b-473e-9110-4c692d8d21f4 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:48:23.331777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-31852f71-6d5b-473e-9110-4c692d8d21f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:23.342958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31852f71-6d5b-473e-9110-4c692d8d21f4 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:48:23.368114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:23.436976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f9991-fd89-431d-8d11-c767a41cdff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:48:23.437609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9991-fd89-431d-8d11-c767a41cdff3 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:23.437883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9991-fd89-431d-8d11-c767a41cdff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:23.438149 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9991-fd89-431d-8d11-c767a41cdff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:23.438781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f9991-fd89-431d-8d11-c767a41cdff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:48:23.439787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WARNING py.warnings [None req-2b4f9991-fd89-431d-8d11-c767a41cdff3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:48:23.439787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: warnings.warn(msg) Oct 03 14:48:23.439787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: Oct 03 14:48:23.440664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f9991-fd89-431d-8d11-c767a41cdff3 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:48:23.453316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 753/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:48:23.457639 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadefffa-51d6-4336-8405-dfbae983a94f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:48:23.459163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-fadefffa-51d6-4336-8405-dfbae983a94f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:23.492223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadefffa-51d6-4336-8405-dfbae983a94f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:48:23.583596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d317b-74b8-4232-983f-8184de8a45be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:48:23.584306 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d317b-74b8-4232-983f-8184de8a45be admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:23.584574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d317b-74b8-4232-983f-8184de8a45be admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:23.584978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d317b-74b8-4232-983f-8184de8a45be admin admin] PATH_INFO: `/v3/projects/8f1e1f286e6a47c586a1e95c2fd01899/users/9d28f79d48a24e9ab5f7f7a72884b8b0/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:23.607335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d317b-74b8-4232-983f-8184de8a45be admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f1e1f286e6a47c586a1e95c2fd01899, user_id=9d28f79d48a24e9ab5f7f7a72884b8b0, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:48:23.608364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: WARNING py.warnings [None req-d25d317b-74b8-4232-983f-8184de8a45be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:48:23.608364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: warnings.warn(msg) Oct 03 14:48:23.608364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: Oct 03 14:48:23.610345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d317b-74b8-4232-983f-8184de8a45be admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:48:23.635809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 748/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:48:23 2022] PUT /identity/v3/projects/8f1e1f286e6a47c586a1e95c2fd01899/users/9d28f79d48a24e9ab5f7f7a72884b8b0/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:48:23.648116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc993803-5a1c-46b9-99f7-61cb65df486b None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:23.648402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc993803-5a1c-46b9-99f7-61cb65df486b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:23.648675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc993803-5a1c-46b9-99f7-61cb65df486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:23.706274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-fc993803-5a1c-46b9-99f7-61cb65df486b None None] MFA Rules not processed for user `9d28f79d48a24e9ab5f7f7a72884b8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:48:23.707400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-fc993803-5a1c-46b9-99f7-61cb65df486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:23.782112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 754/1502] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:48:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:48:23.795849 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eafef-4019-4936-9c0e-2ea36773a5bd None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:23.796145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eafef-4019-4936-9c0e-2ea36773a5bd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:23.796452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eafef-4019-4936-9c0e-2ea36773a5bd None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:23.797855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 749/1503] 10.222.0.46 () {54 vars in 963 bytes} [Mon Oct 3 14:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:48:23.802534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdc4fc-623e-4028-9768-b42237d0b5be None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:23.802817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdc4fc-623e-4028-9768-b42237d0b5be None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:23.803088 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdc4fc-623e-4028-9768-b42237d0b5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:23.834334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:23.878450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-fabdc4fc-623e-4028-9768-b42237d0b5be None None] MFA Rules not processed for user `64a9421bec3b4b9e89dec35d7556d025`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:48:23.879631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-fabdc4fc-623e-4028-9768-b42237d0b5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:23.952012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 755/1504] 10.222.0.46 () {58 vars in 1060 bytes} [Mon Oct 3 14:48:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:48:23.960144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eafef-4019-4936-9c0e-2ea36773a5bd None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:48:23.961742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5b4eafef-4019-4936-9c0e-2ea36773a5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:24.012868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:24.066505 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eafef-4019-4936-9c0e-2ea36773a5bd None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:48:24.136945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ae7e-5d9a-4e28-b9a7-df58fddc361b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:48:24.137563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ae7e-5d9a-4e28-b9a7-df58fddc361b service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:24.137838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ae7e-5d9a-4e28-b9a7-df58fddc361b service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:24.138105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ae7e-5d9a-4e28-b9a7-df58fddc361b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:24.139679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d050ae7e-5d9a-4e28-b9a7-df58fddc361b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:24.234817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ae7e-5d9a-4e28-b9a7-df58fddc361b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:48:24.236165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ae7e-5d9a-4e28-b9a7-df58fddc361b service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:48:24.239559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:24.255797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 750/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:48:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:48:24.301817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d89ca-1c5d-4fe6-aa92-188b4c643441 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:24.302117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d89ca-1c5d-4fe6-aa92-188b4c643441 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:24.302368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d89ca-1c5d-4fe6-aa92-188b4c643441 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:24.303647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 756/1506] 10.222.0.46 () {52 vars in 830 bytes} [Mon Oct 3 14:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:48:24.307188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ae7e-5d9a-4e28-b9a7-df58fddc361b service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:48:24.308449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d050ae7e-5d9a-4e28-b9a7-df58fddc361b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:24.409913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ae7e-5d9a-4e28-b9a7-df58fddc361b service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:48:24.500496 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:48:24.501060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:24.501334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:24.501631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] PATH_INFO: `/v3/users/9d28f79d48a24e9ab5f7f7a72884b8b0` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:24.515782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d28f79d48a24e9ab5f7f7a72884b8b0)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:48:24.517369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: WARNING py.warnings [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:48:24.517369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: warnings.warn(msg) Oct 03 14:48:24.517369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: Oct 03 14:48:24.519574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:48:24.534267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:24.592497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d28f79d48a24e9ab5f7f7a72884b8b0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:48:24.593142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d28f79d48a24e9ab5f7f7a72884b8b0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:48:24.607486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d28f79d48a24e9ab5f7f7a72884b8b0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:48:24.632927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d28f79d48a24e9ab5f7f7a72884b8b0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:48:24.643673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d28f79d48a24e9ab5f7f7a72884b8b0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:48:24.651368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 751/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:48:24 2022] DELETE /identity/v3/users/9d28f79d48a24e9ab5f7f7a72884b8b0 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:48:24.656009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bad96-6f44-477c-b96b-b791795dc147 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:24.656572 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bad96-6f44-477c-b96b-b791795dc147 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:24.656719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bad96-6f44-477c-b96b-b791795dc147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:24.721980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-652bad96-6f44-477c-b96b-b791795dc147 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:48:24.723188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-652bad96-6f44-477c-b96b-b791795dc147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:24.795780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:24.808808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 757/1508] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:48:24.818405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:48:24.820218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:24.918466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6791c6-d300-498d-b66e-b39506fb99a0 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:48:24.971545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:24.991561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1ae80-5e32-4bb6-a1a1-6c3d033023d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:48:24.992175 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1ae80-5e32-4bb6-a1a1-6c3d033023d3 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:24.992448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1ae80-5e32-4bb6-a1a1-6c3d033023d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:24.992716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1ae80-5e32-4bb6-a1a1-6c3d033023d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:24.994390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-13d1ae80-5e32-4bb6-a1a1-6c3d033023d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:25.100503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1ae80-5e32-4bb6-a1a1-6c3d033023d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:48:25.101911 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1ae80-5e32-4bb6-a1a1-6c3d033023d3 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:48:25.121728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 752/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:48:25.475298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bad96-6f44-477c-b96b-b791795dc147 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:48:25.476551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-652bad96-6f44-477c-b96b-b791795dc147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:48:25.487812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bad96-6f44-477c-b96b-b791795dc147 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:48:25.583482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:48:25.584108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:48:25.584382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:48:25.584683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] PATH_INFO: `/v3/projects/8f1e1f286e6a47c586a1e95c2fd01899` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:48:25.595112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f1e1f286e6a47c586a1e95c2fd01899)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:48:25.596135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WARNING py.warnings [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:48:25.596135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: warnings.warn(msg) Oct 03 14:48:25.596135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: Oct 03 14:48:25.597326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:48:25.654663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f1e1f286e6a47c586a1e95c2fd01899', 'request_id': 'req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:48:25.663579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f1e1f286e6a47c586a1e95c2fd01899', 'request_id': 'req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:48:25.676841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f1e1f286e6a47c586a1e95c2fd01899', 'request_id': 'req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:48:25.699841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] The token cache is being invalidate because project 8f1e1f286e6a47c586a1e95c2fd01899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:48:25.700129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:48:25.700555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7be69a6a-12ee-44b2-b9d4-00f523ab4b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:48:25.701999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 758/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:48:25 2022] DELETE /identity/v3/projects/8f1e1f286e6a47c586a1e95c2fd01899 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:48:35.119520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=62 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:48:35.697934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:49:52.468248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104106c-e48b-4a0a-8e30-ef34a25a0085 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:49:52.468838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104106c-e48b-4a0a-8e30-ef34a25a0085 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:49:52.468838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104106c-e48b-4a0a-8e30-ef34a25a0085 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:49:52.470195 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:49:52.475446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af771e98-9d74-4e18-8a95-f67490f51778 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:49:52.475858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af771e98-9d74-4e18-8a95-f67490f51778 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:49:52.475980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af771e98-9d74-4e18-8a95-f67490f51778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:49:52.554798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-af771e98-9d74-4e18-8a95-f67490f51778 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:49:52.555933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-af771e98-9d74-4e18-8a95-f67490f51778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:49:52.662078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:49:52.815622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104106c-e48b-4a0a-8e30-ef34a25a0085 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:49:52.818242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e104106c-e48b-4a0a-8e30-ef34a25a0085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:49:52.936123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104106c-e48b-4a0a-8e30-ef34a25a0085 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:49:52.941553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d53e2-8449-4d58-8134-d24d7887820b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:49:52.942157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d53e2-8449-4d58-8134-d24d7887820b service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:49:52.943013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d53e2-8449-4d58-8134-d24d7887820b service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:49:52.943281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d53e2-8449-4d58-8134-d24d7887820b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:49:52.951298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d53e2-8449-4d58-8134-d24d7887820b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:49:52.952899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d53e2-8449-4d58-8134-d24d7887820b service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:49:52.962013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 754/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:49:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:49:54.034197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42664f52-a2b1-4d51-8ddf-7630631be543 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:49:54.034635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42664f52-a2b1-4d51-8ddf-7630631be543 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:49:54.034737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42664f52-a2b1-4d51-8ddf-7630631be543 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:49:54.036134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:49:54.042496 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28020c7-3263-4de7-a81a-d1bdf97beaf7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:49:54.042961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28020c7-3263-4de7-a81a-d1bdf97beaf7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:49:54.043055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28020c7-3263-4de7-a81a-d1bdf97beaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:49:54.062452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e28020c7-3263-4de7-a81a-d1bdf97beaf7 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:49:54.063644 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e28020c7-3263-4de7-a81a-d1bdf97beaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:49:54.071875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:49:54.222668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42664f52-a2b1-4d51-8ddf-7630631be543 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:49:54.236695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42664f52-a2b1-4d51-8ddf-7630631be543 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:49:54.242638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ea8ba-e32f-4630-a260-c5c23b187462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:49:54.243233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ea8ba-e32f-4630-a260-c5c23b187462 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:49:54.243509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ea8ba-e32f-4630-a260-c5c23b187462 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:49:54.243781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ea8ba-e32f-4630-a260-c5c23b187462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:49:54.252393 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ea8ba-e32f-4630-a260-c5c23b187462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:49:54.254089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ea8ba-e32f-4630-a260-c5c23b187462 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:49:54.264013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 761/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:49:54.316579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b97ff-b0be-4d94-b78a-1a472a22f19d None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:49:54.317574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b97ff-b0be-4d94-b78a-1a472a22f19d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:49:54.318005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b97ff-b0be-4d94-b78a-1a472a22f19d None None] PATH_INFO: `/v3` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:49:54.319360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 756/1517] 10.222.0.42 () {54 vars in 925 bytes} [Mon Oct 3 14:49:54 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:49:54.326482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9c8a3-5cfc-4d52-b168-97f7f03ef842 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:49:54.326769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9c8a3-5cfc-4d52-b168-97f7f03ef842 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:49:54.327201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9c8a3-5cfc-4d52-b168-97f7f03ef842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:49:54.394730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b6c9c8a3-5cfc-4d52-b168-97f7f03ef842 None None] MFA Rules not processed for user `ebc01c87551044818e1ccbb46b8e7340`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:49:54.395949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b6c9c8a3-5cfc-4d52-b168-97f7f03ef842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:49:54.413083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:49:54.467515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 762/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Oct 3 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:50:03.906239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571295a-7484-4719-8b8e-03712fb8768e None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:03.906674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571295a-7484-4719-8b8e-03712fb8768e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:03.906783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571295a-7484-4719-8b8e-03712fb8768e None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:03.908128 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 757/1519] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 3 14:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:50:03.913742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21807e-d383-41be-8c0f-246ee160c8c0 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:03.914186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21807e-d383-41be-8c0f-246ee160c8c0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:03.914290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21807e-d383-41be-8c0f-246ee160c8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:03.934292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-8a21807e-d383-41be-8c0f-246ee160c8c0 None None] MFA Rules not processed for user `ebc01c87551044818e1ccbb46b8e7340`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:50:03.935470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8a21807e-d383-41be-8c0f-246ee160c8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:50:03.944229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 763/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 3 14:50:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:50:03.951822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571295a-7484-4719-8b8e-03712fb8768e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:50:03.964646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571295a-7484-4719-8b8e-03712fb8768e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:50:03.969459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626acc1-baf2-4fbd-b856-c1b71848503a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:50:03.970060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626acc1-baf2-4fbd-b856-c1b71848503a service placement] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:03.970337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626acc1-baf2-4fbd-b856-c1b71848503a service placement] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:03.970605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626acc1-baf2-4fbd-b856-c1b71848503a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:03.979230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626acc1-baf2-4fbd-b856-c1b71848503a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:50:03.980793 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626acc1-baf2-4fbd-b856-c1b71848503a service placement] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:50:03.990643 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 758/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:50:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:50:05.759344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b4b6d-1b70-4a78-98ce-9cd12b77e9e0 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:05.759910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b4b6d-1b70-4a78-98ce-9cd12b77e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:05.759910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b4b6d-1b70-4a78-98ce-9cd12b77e9e0 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:05.761224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:50:05.767358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80797ef-d956-4e13-a8f8-9d98933136f7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:05.767796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80797ef-d956-4e13-a8f8-9d98933136f7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:05.767899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80797ef-d956-4e13-a8f8-9d98933136f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:05.786951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e80797ef-d956-4e13-a8f8-9d98933136f7 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:50:05.788146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e80797ef-d956-4e13-a8f8-9d98933136f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:50:05.795950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:50:05.948933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b4b6d-1b70-4a78-98ce-9cd12b77e9e0 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:50:05.963480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b4b6d-1b70-4a78-98ce-9cd12b77e9e0 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:50:05.968722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03de1d-b5e2-4c43-9134-2f657387b381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:50:05.969339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03de1d-b5e2-4c43-9134-2f657387b381 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:05.969654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03de1d-b5e2-4c43-9134-2f657387b381 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:05.969926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03de1d-b5e2-4c43-9134-2f657387b381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:05.978953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03de1d-b5e2-4c43-9134-2f657387b381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:50:05.980601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03de1d-b5e2-4c43-9134-2f657387b381 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:50:05.990408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 765/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:50:07.061938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00c732-518c-41e3-84c3-a9e53bdbca58 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:07.062515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00c732-518c-41e3-84c3-a9e53bdbca58 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:07.062515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00c732-518c-41e3-84c3-a9e53bdbca58 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:07.063894 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 14:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:50:07.070227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3696212-a4a5-4f3f-ae60-b1be17f4e83f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:07.070676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3696212-a4a5-4f3f-ae60-b1be17f4e83f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:07.070799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3696212-a4a5-4f3f-ae60-b1be17f4e83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:07.090270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c3696212-a4a5-4f3f-ae60-b1be17f4e83f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:50:07.091461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c3696212-a4a5-4f3f-ae60-b1be17f4e83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:50:07.100060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 14:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:50:07.249785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00c732-518c-41e3-84c3-a9e53bdbca58 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:50:07.263982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00c732-518c-41e3-84c3-a9e53bdbca58 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:50:07.269345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd6ec5-23df-471a-91e2-3334d4cc890e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:50:07.269944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd6ec5-23df-471a-91e2-3334d4cc890e service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:07.270227 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd6ec5-23df-471a-91e2-3334d4cc890e service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:07.270992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd6ec5-23df-471a-91e2-3334d4cc890e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:07.279149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd6ec5-23df-471a-91e2-3334d4cc890e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:50:07.280770 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd6ec5-23df-471a-91e2-3334d4cc890e service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:50:07.292027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 761/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:50:11.196308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32035824-3587-4944-ab10-fdc3af24a761 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:11.196887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32035824-3587-4944-ab10-fdc3af24a761 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:11.196887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32035824-3587-4944-ab10-fdc3af24a761 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:11.198224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:50:11.203506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92064c-e326-4661-8f63-1987486df5d9 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:11.203921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92064c-e326-4661-8f63-1987486df5d9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:11.204062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92064c-e326-4661-8f63-1987486df5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:11.223905 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-5d92064c-e326-4661-8f63-1987486df5d9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:50:11.225097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5d92064c-e326-4661-8f63-1987486df5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:50:11.232814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:50:11.524511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32035824-3587-4944-ab10-fdc3af24a761 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:50:11.526216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-32035824-3587-4944-ab10-fdc3af24a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:50:11.603506 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32035824-3587-4944-ab10-fdc3af24a761 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:50:11.606939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aadaa8-1dc8-4d76-8fe2-a9cba0c50e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:50:11.607562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aadaa8-1dc8-4d76-8fe2-a9cba0c50e9d service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:11.607834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aadaa8-1dc8-4d76-8fe2-a9cba0c50e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:11.608100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aadaa8-1dc8-4d76-8fe2-a9cba0c50e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:11.616753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aadaa8-1dc8-4d76-8fe2-a9cba0c50e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:50:11.618208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aadaa8-1dc8-4d76-8fe2-a9cba0c50e9d service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:50:11.627384 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 768/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:50:13.766983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98e55f-9d81-4c4e-8982-819055b1e0bd None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:13.767554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98e55f-9d81-4c4e-8982-819055b1e0bd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:13.767554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98e55f-9d81-4c4e-8982-819055b1e0bd None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:13.768873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:50:13.774646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ace1a-7b2b-4846-abb5-bcd695e70d80 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:13.775082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ace1a-7b2b-4846-abb5-bcd695e70d80 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:13.775205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ace1a-7b2b-4846-abb5-bcd695e70d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:13.794322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-557ace1a-7b2b-4846-abb5-bcd695e70d80 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:50:13.795507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-557ace1a-7b2b-4846-abb5-bcd695e70d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:50:13.804143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:50:14.097977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98e55f-9d81-4c4e-8982-819055b1e0bd None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:50:14.108656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98e55f-9d81-4c4e-8982-819055b1e0bd None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:50:14.112539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccaa87d-ef68-4a5a-acf2-fbd8ea61c137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:50:14.113129 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccaa87d-ef68-4a5a-acf2-fbd8ea61c137 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:14.113403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccaa87d-ef68-4a5a-acf2-fbd8ea61c137 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:14.114242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccaa87d-ef68-4a5a-acf2-fbd8ea61c137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:14.123232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccaa87d-ef68-4a5a-acf2-fbd8ea61c137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:50:14.124658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccaa87d-ef68-4a5a-acf2-fbd8ea61c137 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:50:14.134497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 764/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:50:17.724992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1d151-444b-49e9-8ee4-040face7105e None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:17.725620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1d151-444b-49e9-8ee4-040face7105e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:17.725620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1d151-444b-49e9-8ee4-040face7105e None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:17.726896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:50:17.732297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4c94e-8cea-43cc-a67a-0638428e33b7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:17.732729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4c94e-8cea-43cc-a67a-0638428e33b7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:17.732887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4c94e-8cea-43cc-a67a-0638428e33b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:17.740892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=81 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:50:17.753451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-73c4c94e-8cea-43cc-a67a-0638428e33b7 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:50:17.755304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-73c4c94e-8cea-43cc-a67a-0638428e33b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:50:17.763339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:50:18.063775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1d151-444b-49e9-8ee4-040face7105e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:50:18.065404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-34f1d151-444b-49e9-8ee4-040face7105e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:50:18.128732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:50:18.143339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1d151-444b-49e9-8ee4-040face7105e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:50:18.146942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89c60a-1c40-48fd-8974-945b96da3fa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:50:18.147543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89c60a-1c40-48fd-8974-945b96da3fa4 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:18.147821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89c60a-1c40-48fd-8974-945b96da3fa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:18.148091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89c60a-1c40-48fd-8974-945b96da3fa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:18.157097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89c60a-1c40-48fd-8974-945b96da3fa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:50:18.158565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89c60a-1c40-48fd-8974-945b96da3fa4 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:50:18.168283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 771/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:50:20.299271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd4d29-4ace-4172-9e81-40623f4ef35b None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:20.299740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd4d29-4ace-4172-9e81-40623f4ef35b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:20.299844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd4d29-4ace-4172-9e81-40623f4ef35b None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:20.301160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 14:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:50:20.306968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9adb1e-b2b3-4398-9920-81cb8becdcec None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:20.307395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9adb1e-b2b3-4398-9920-81cb8becdcec None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:20.307504 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9adb1e-b2b3-4398-9920-81cb8becdcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:20.327146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-8c9adb1e-b2b3-4398-9920-81cb8becdcec None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:50:20.328341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8c9adb1e-b2b3-4398-9920-81cb8becdcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:50:20.336195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:50:20.641042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd4d29-4ace-4172-9e81-40623f4ef35b None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:50:20.642766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-cfdd4d29-4ace-4172-9e81-40623f4ef35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:50:20.655363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd4d29-4ace-4172-9e81-40623f4ef35b None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:50:20.658808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fe06c-7255-4fe4-a791-349724cd2b67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:50:20.659397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fe06c-7255-4fe4-a791-349724cd2b67 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:50:20.659675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fe06c-7255-4fe4-a791-349724cd2b67 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:50:20.659948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fe06c-7255-4fe4-a791-349724cd2b67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:50:20.668564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fe06c-7255-4fe4-a791-349724cd2b67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:50:20.670054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fe06c-7255-4fe4-a791-349724cd2b67 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:50:20.679660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 767/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:50:30.322349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=11 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:50:30.675109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=8 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:44.523482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948765a6-8e9d-44bb-a9f5-9c66ac19a787 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.524667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948765a6-8e9d-44bb-a9f5-9c66ac19a787 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.525268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948765a6-8e9d-44bb-a9f5-9c66ac19a787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.548899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-948765a6-8e9d-44bb-a9f5-9c66ac19a787 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:44.550215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-948765a6-8e9d-44bb-a9f5-9c66ac19a787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:44.562672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863a44e-e2f0-41ce-80bd-d1898d5801d8 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.563708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863a44e-e2f0-41ce-80bd-d1898d5801d8 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.563708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863a44e-e2f0-41ce-80bd-d1898d5801d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.565073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 773/1540] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:44.574567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d758933-f17d-4047-91d0-95482e1d8a8f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.574898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d758933-f17d-4047-91d0-95482e1d8a8f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.575112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d758933-f17d-4047-91d0-95482e1d8a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.595220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-1863a44e-e2f0-41ce-80bd-d1898d5801d8 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:44.596463 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1863a44e-e2f0-41ce-80bd-d1898d5801d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:44.602730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-0d758933-f17d-4047-91d0-95482e1d8a8f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:44.603948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-0d758933-f17d-4047-91d0-95482e1d8a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:44.608946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 768/1541] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:44.617361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31d81-95be-47a1-a725-5b6adfc3e598 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.617721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31d81-95be-47a1-a725-5b6adfc3e598 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.617959 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31d81-95be-47a1-a725-5b6adfc3e598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.619080 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 774/1542] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:51:44.623499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d758933-f17d-4047-91d0-95482e1d8a8f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:44.643488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d758933-f17d-4047-91d0-95482e1d8a8f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:44.645839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-74a31d81-95be-47a1-a725-5b6adfc3e598 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:44.647612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-74a31d81-95be-47a1-a725-5b6adfc3e598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:44.649252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9c347-2fb2-498d-b3e9-ae536e80f85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:44.649911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9c347-2fb2-498d-b3e9-ae536e80f85d admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.650190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9c347-2fb2-498d-b3e9-ae536e80f85d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.650460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9c347-2fb2-498d-b3e9-ae536e80f85d admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.651118 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9c347-2fb2-498d-b3e9-ae536e80f85d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:44.651491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9c347-2fb2-498d-b3e9-ae536e80f85d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:44.653100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9c347-2fb2-498d-b3e9-ae536e80f85d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:44.672201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 769/1543] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:51:44.676983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31d81-95be-47a1-a725-5b6adfc3e598 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:44.678963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 775/1544] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:51:44.691224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9c347-2fb2-498d-b3e9-ae536e80f85d admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:44.693032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31d81-95be-47a1-a725-5b6adfc3e598 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:44.707736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8f14d-6af7-4678-a8e9-68b717ffcde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:44.708378 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8f14d-6af7-4678-a8e9-68b717ffcde2 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.708665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8f14d-6af7-4678-a8e9-68b717ffcde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.709744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8f14d-6af7-4678-a8e9-68b717ffcde2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.710087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9c347-2fb2-498d-b3e9-ae536e80f85d admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:44.710403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8f14d-6af7-4678-a8e9-68b717ffcde2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:44.710804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8f14d-6af7-4678-a8e9-68b717ffcde2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:44.711977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: WARNING py.warnings [None req-b4b8f14d-6af7-4678-a8e9-68b717ffcde2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:51:44.711977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: warnings.warn(msg) Oct 03 14:51:44.711977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: Oct 03 14:51:44.712897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8f14d-6af7-4678-a8e9-68b717ffcde2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:44.722734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f22a9-3d66-4576-9ab2-e5e309923449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:44.723365 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f22a9-3d66-4576-9ab2-e5e309923449 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.723648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f22a9-3d66-4576-9ab2-e5e309923449 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.724222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f22a9-3d66-4576-9ab2-e5e309923449 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.725450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f22a9-3d66-4576-9ab2-e5e309923449 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:44.727484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 770/1545] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:51:44.730573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WARNING py.warnings [None req-b08f22a9-3d66-4576-9ab2-e5e309923449 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:51:44.730573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: warnings.warn(msg) Oct 03 14:51:44.730573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: Oct 03 14:51:44.731793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f22a9-3d66-4576-9ab2-e5e309923449 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:44.737148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8f14d-6af7-4678-a8e9-68b717ffcde2 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:44.750171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8f14d-6af7-4678-a8e9-68b717ffcde2 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:44.764428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d082f79-4287-419b-90da-825a4727a121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:44.765066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082f79-4287-419b-90da-825a4727a121 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.765355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082f79-4287-419b-90da-825a4727a121 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.765670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082f79-4287-419b-90da-825a4727a121 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.766315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d082f79-4287-419b-90da-825a4727a121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:44.768716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d082f79-4287-419b-90da-825a4727a121 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:44.799963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 776/1546] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:44.801412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28985c3-5ef7-41f3-83b4-d9833719d498 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.801731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28985c3-5ef7-41f3-83b4-d9833719d498 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.802003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28985c3-5ef7-41f3-83b4-d9833719d498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.808156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 771/1547] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/projects => generated 391 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:44.809733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad917acb-3e2f-4c51-bee0-8b02df01b264 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.810029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad917acb-3e2f-4c51-bee0-8b02df01b264 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.810309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad917acb-3e2f-4c51-bee0-8b02df01b264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.827309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e28985c3-5ef7-41f3-83b4-d9833719d498 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:44.829045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e28985c3-5ef7-41f3-83b4-d9833719d498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:44.833563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ad917acb-3e2f-4c51-bee0-8b02df01b264 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:44.835448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ad917acb-3e2f-4c51-bee0-8b02df01b264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:44.897669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 777/1548] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:44.897669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856e0a8-f712-4bf7-b0db-aeb5fdc1141a None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.897669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856e0a8-f712-4bf7-b0db-aeb5fdc1141a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.897669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856e0a8-f712-4bf7-b0db-aeb5fdc1141a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.904816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 772/1549] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:44.906175 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd253a0-f826-481c-a3a6-2863bfaa5a21 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.906462 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd253a0-f826-481c-a3a6-2863bfaa5a21 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.906733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd253a0-f826-481c-a3a6-2863bfaa5a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.916259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-6856e0a8-f712-4bf7-b0db-aeb5fdc1141a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:44.917454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6856e0a8-f712-4bf7-b0db-aeb5fdc1141a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:44.925332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 778/1550] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:44.926120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856e0a8-f712-4bf7-b0db-aeb5fdc1141a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:44.927362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-1cd253a0-f826-481c-a3a6-2863bfaa5a21 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:44.928570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1cd253a0-f826-481c-a3a6-2863bfaa5a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:44.935396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856e0a8-f712-4bf7-b0db-aeb5fdc1141a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:44.936603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 773/1551] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:44.937697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd253a0-f826-481c-a3a6-2863bfaa5a21 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:44.941081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a10bec-e78f-4fcb-b682-29576bab85b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:44.941706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a10bec-e78f-4fcb-b682-29576bab85b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.941985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a10bec-e78f-4fcb-b682-29576bab85b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.942252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a10bec-e78f-4fcb-b682-29576bab85b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.942876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a10bec-e78f-4fcb-b682-29576bab85b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:44.945360 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a10bec-e78f-4fcb-b682-29576bab85b7 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:44.952324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd253a0-f826-481c-a3a6-2863bfaa5a21 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:44.956920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba7e7b-7a9a-4488-883c-8f6b89cc198c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:44.957584 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba7e7b-7a9a-4488-883c-8f6b89cc198c admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.957944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba7e7b-7a9a-4488-883c-8f6b89cc198c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.958335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba7e7b-7a9a-4488-883c-8f6b89cc198c admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.959584 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba7e7b-7a9a-4488-883c-8f6b89cc198c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:44.960599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: WARNING py.warnings [None req-d7ba7e7b-7a9a-4488-883c-8f6b89cc198c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:51:44.960599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: warnings.warn(msg) Oct 03 14:51:44.960599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: Oct 03 14:51:44.961841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba7e7b-7a9a-4488-883c-8f6b89cc198c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:44.968639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 779/1552] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:44.971287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da287a-b736-4012-82ed-75356fc642bd None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.971566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da287a-b736-4012-82ed-75356fc642bd None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.971929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da287a-b736-4012-82ed-75356fc642bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.989755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 774/1553] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/users => generated 296 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:44.989755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4db135-c77d-4d96-886b-02478823afeb None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:44.989755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4db135-c77d-4d96-886b-02478823afeb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:44.989755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4db135-c77d-4d96-886b-02478823afeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:44.992323 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-96da287a-b736-4012-82ed-75356fc642bd None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:44.993538 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-96da287a-b736-4012-82ed-75356fc642bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:45.001271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 780/1554] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:45.002450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a9197-570e-434a-a0ea-8c783c6b8757 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.002740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a9197-570e-434a-a0ea-8c783c6b8757 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.003010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a9197-570e-434a-a0ea-8c783c6b8757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.008148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-8c4db135-c77d-4d96-886b-02478823afeb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:45.009340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8c4db135-c77d-4d96-886b-02478823afeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:45.017384 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 775/1555] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:45.018604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4907bae-a2bd-44b5-a8d6-fb348aeff84b None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.018885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4907bae-a2bd-44b5-a8d6-fb348aeff84b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.019153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4907bae-a2bd-44b5-a8d6-fb348aeff84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.024123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e52a9197-570e-434a-a0ea-8c783c6b8757 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:45.025351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e52a9197-570e-434a-a0ea-8c783c6b8757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:45.033351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 781/1556] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:45.034544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbbef3-f58a-4af2-ad7e-ce432fdbbfac None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.034827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbbef3-f58a-4af2-ad7e-ce432fdbbfac None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.035219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbbef3-f58a-4af2-ad7e-ce432fdbbfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.039599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e4907bae-a2bd-44b5-a8d6-fb348aeff84b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:45.041132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e4907bae-a2bd-44b5-a8d6-fb348aeff84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:45.050356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 776/1557] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:45.051631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8455cc-11b8-4e89-afbb-bf62f7944fee None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.051922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8455cc-11b8-4e89-afbb-bf62f7944fee None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.052198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8455cc-11b8-4e89-afbb-bf62f7944fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.054065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b9dbbef3-f58a-4af2-ad7e-ce432fdbbfac None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:45.055446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b9dbbef3-f58a-4af2-ad7e-ce432fdbbfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:45.063168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 782/1558] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:45.064012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbbef3-f58a-4af2-ad7e-ce432fdbbfac None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.072227 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ab8455cc-11b8-4e89-afbb-bf62f7944fee None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:45.073648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ab8455cc-11b8-4e89-afbb-bf62f7944fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:45.074160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbbef3-f58a-4af2-ad7e-ce432fdbbfac None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.078670 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fe2cb-228d-49e0-b801-ca035edc010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.079248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe2cb-228d-49e0-b801-ca035edc010c admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.079520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe2cb-228d-49e0-b801-ca035edc010c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.079784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe2cb-228d-49e0-b801-ca035edc010c admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.080438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fe2cb-228d-49e0-b801-ca035edc010c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.081533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 777/1559] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:45.082030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fe2cb-228d-49e0-b801-ca035edc010c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.086369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8455cc-11b8-4e89-afbb-bf62f7944fee None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.092753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 783/1560] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:45.095983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260ff97-2cce-4d6d-a211-c43347c158ee None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.095983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260ff97-2cce-4d6d-a211-c43347c158ee None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.095983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260ff97-2cce-4d6d-a211-c43347c158ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.096215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8455cc-11b8-4e89-afbb-bf62f7944fee None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.100673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f2f60-51a6-4a9b-905b-d3089342cb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.101262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f2f60-51a6-4a9b-905b-d3089342cb59 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.101565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f2f60-51a6-4a9b-905b-d3089342cb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.101839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f2f60-51a6-4a9b-905b-d3089342cb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.102493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f2f60-51a6-4a9b-905b-d3089342cb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.104071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f2f60-51a6-4a9b-905b-d3089342cb59 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.113208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d260ff97-2cce-4d6d-a211-c43347c158ee None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:45.116493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 778/1561] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:45.116643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d260ff97-2cce-4d6d-a211-c43347c158ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:45.116742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f2f60-51a6-4a9b-905b-d3089342cb59 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.123356 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 784/1562] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:45.124722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7daccc-c619-422a-9bb9-2a2caff1805f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.125009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7daccc-c619-422a-9bb9-2a2caff1805f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.125427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7daccc-c619-422a-9bb9-2a2caff1805f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.126064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f2f60-51a6-4a9b-905b-d3089342cb59 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.131171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8d4b0-d0a6-4cbd-abb6-6167f245a3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.131753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8d4b0-d0a6-4cbd-abb6-6167f245a3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.132037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8d4b0-d0a6-4cbd-abb6-6167f245a3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.132310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8d4b0-d0a6-4cbd-abb6-6167f245a3e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.132949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8d4b0-d0a6-4cbd-abb6-6167f245a3e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:45.133248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8d4b0-d0a6-4cbd-abb6-6167f245a3e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.134892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8d4b0-d0a6-4cbd-abb6-6167f245a3e4 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.144819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5e7daccc-c619-422a-9bb9-2a2caff1805f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:45.146037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5e7daccc-c619-422a-9bb9-2a2caff1805f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:45.147111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 779/1563] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:51:45.147895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8d4b0-d0a6-4cbd-abb6-6167f245a3e4 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.154813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 785/1564] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:45.155447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7daccc-c619-422a-9bb9-2a2caff1805f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.160292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8d4b0-d0a6-4cbd-abb6-6167f245a3e4 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.164564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7daccc-c619-422a-9bb9-2a2caff1805f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.164870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc62ee-661b-4585-815b-d23b9dade3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.165533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc62ee-661b-4585-815b-d23b9dade3ce admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.165804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc62ee-661b-4585-815b-d23b9dade3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.166074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc62ee-661b-4585-815b-d23b9dade3ce admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.166734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc62ee-661b-4585-815b-d23b9dade3ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:45.167034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc62ee-661b-4585-815b-d23b9dade3ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.168761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc62ee-661b-4585-815b-d23b9dade3ce admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.169621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1a83e-7228-4b38-adb1-968e2b7858a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.170249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1a83e-7228-4b38-adb1-968e2b7858a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.170533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1a83e-7228-4b38-adb1-968e2b7858a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.170805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1a83e-7228-4b38-adb1-968e2b7858a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.171458 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1a83e-7228-4b38-adb1-968e2b7858a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:45.171760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1a83e-7228-4b38-adb1-968e2b7858a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.173347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1a83e-7228-4b38-adb1-968e2b7858a4 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.180667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 780/1565] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:51:45.181472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc62ee-661b-4585-815b-d23b9dade3ce admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.191533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 786/1566] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:51:45.192409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1a83e-7228-4b38-adb1-968e2b7858a4 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.192528 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc62ee-661b-4585-815b-d23b9dade3ce admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.197583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f68e6-bd2e-4b9a-b89f-24ef530fce1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.198170 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f68e6-bd2e-4b9a-b89f-24ef530fce1a admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.198445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f68e6-bd2e-4b9a-b89f-24ef530fce1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.198712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f68e6-bd2e-4b9a-b89f-24ef530fce1a admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.199748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f68e6-bd2e-4b9a-b89f-24ef530fce1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:45.200243 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f68e6-bd2e-4b9a-b89f-24ef530fce1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.201843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f68e6-bd2e-4b9a-b89f-24ef530fce1a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.204180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1a83e-7228-4b38-adb1-968e2b7858a4 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.208692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f036bd-777c-4cbc-9c85-2f3ba99e2044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.209306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f036bd-777c-4cbc-9c85-2f3ba99e2044 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.209615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f036bd-777c-4cbc-9c85-2f3ba99e2044 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.209889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f036bd-777c-4cbc-9c85-2f3ba99e2044 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.210524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f036bd-777c-4cbc-9c85-2f3ba99e2044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.212099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f036bd-777c-4cbc-9c85-2f3ba99e2044 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.213799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 781/1567] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:51:45.214598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f68e6-bd2e-4b9a-b89f-24ef530fce1a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.222789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 787/1568] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:45.226089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f036bd-777c-4cbc-9c85-2f3ba99e2044 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.230508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f68e6-bd2e-4b9a-b89f-24ef530fce1a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.234936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9098a-f2a7-460f-9af6-1a5c128c64f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.235470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f036bd-777c-4cbc-9c85-2f3ba99e2044 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.235588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9098a-f2a7-460f-9af6-1a5c128c64f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.235784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9098a-f2a7-460f-9af6-1a5c128c64f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.236054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9098a-f2a7-460f-9af6-1a5c128c64f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.236717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9098a-f2a7-460f-9af6-1a5c128c64f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.238442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9098a-f2a7-460f-9af6-1a5c128c64f3 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.240700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2600a2c-f0e9-4b17-8da5-d1d1c3940327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.241281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2600a2c-f0e9-4b17-8da5-d1d1c3940327 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.241596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2600a2c-f0e9-4b17-8da5-d1d1c3940327 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.241872 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2600a2c-f0e9-4b17-8da5-d1d1c3940327 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.242544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2600a2c-f0e9-4b17-8da5-d1d1c3940327 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:45.242846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2600a2c-f0e9-4b17-8da5-d1d1c3940327 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.244411 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2600a2c-f0e9-4b17-8da5-d1d1c3940327 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.248781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 782/1569] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:45.249572 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9098a-f2a7-460f-9af6-1a5c128c64f3 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.251546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:45.264960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 788/1570] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:51:45.264960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2600a2c-f0e9-4b17-8da5-d1d1c3940327 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.265240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9098a-f2a7-460f-9af6-1a5c128c64f3 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.265240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f409c-6911-413e-ba22-bfceb17b10f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.265448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f409c-6911-413e-ba22-bfceb17b10f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.265825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f409c-6911-413e-ba22-bfceb17b10f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.266046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f409c-6911-413e-ba22-bfceb17b10f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.266680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f409c-6911-413e-ba22-bfceb17b10f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.268722 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f409c-6911-413e-ba22-bfceb17b10f5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.273233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:45.274618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2600a2c-f0e9-4b17-8da5-d1d1c3940327 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.283564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9635f-3fff-44c1-ba42-334d1ac4317e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.284154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9635f-3fff-44c1-ba42-334d1ac4317e admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.284430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9635f-3fff-44c1-ba42-334d1ac4317e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.284697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9635f-3fff-44c1-ba42-334d1ac4317e admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.285337 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9635f-3fff-44c1-ba42-334d1ac4317e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:45.285667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9635f-3fff-44c1-ba42-334d1ac4317e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.287293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9635f-3fff-44c1-ba42-334d1ac4317e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.301029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 789/1571] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:51:45.301797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9635f-3fff-44c1-ba42-334d1ac4317e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.303251 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 783/1572] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.303901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f409c-6911-413e-ba22-bfceb17b10f5 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.315141 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9635f-3fff-44c1-ba42-334d1ac4317e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.371152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f409c-6911-413e-ba22-bfceb17b10f5 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.375108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850ebba-f736-4a6c-a3a1-a781209a33b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.375698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adae33e-3a9a-4b63-b3f3-b9b124e621cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.375850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850ebba-f736-4a6c-a3a1-a781209a33b1 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.376057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850ebba-f736-4a6c-a3a1-a781209a33b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.376333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adae33e-3a9a-4b63-b3f3-b9b124e621cf admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.376455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850ebba-f736-4a6c-a3a1-a781209a33b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.376628 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adae33e-3a9a-4b63-b3f3-b9b124e621cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.376860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adae33e-3a9a-4b63-b3f3-b9b124e621cf admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.376973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850ebba-f736-4a6c-a3a1-a781209a33b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.377490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adae33e-3a9a-4b63-b3f3-b9b124e621cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.378872 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850ebba-f736-4a6c-a3a1-a781209a33b1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.379412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adae33e-3a9a-4b63-b3f3-b9b124e621cf admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.406780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 784/1573] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.407600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adae33e-3a9a-4b63-b3f3-b9b124e621cf admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.408822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 790/1574] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.408822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850ebba-f736-4a6c-a3a1-a781209a33b1 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.475808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850ebba-f736-4a6c-a3a1-a781209a33b1 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.476308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adae33e-3a9a-4b63-b3f3-b9b124e621cf admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.480381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388121f-aa73-43a0-b61b-6340469e5193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.481026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388121f-aa73-43a0-b61b-6340469e5193 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.481167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dafe3-c6f4-46ba-830e-2a85e07a32c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.481291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388121f-aa73-43a0-b61b-6340469e5193 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.481590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388121f-aa73-43a0-b61b-6340469e5193 admin admin] PATH_INFO: `/v3/projects/f441c0592cd146d3be851ac96cbc49dc/users/89c747f18b434e469ae61755e9f639ca/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.481698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dafe3-c6f4-46ba-830e-2a85e07a32c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.481969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dafe3-c6f4-46ba-830e-2a85e07a32c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.482247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dafe3-c6f4-46ba-830e-2a85e07a32c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.483081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dafe3-c6f4-46ba-830e-2a85e07a32c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.485022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dafe3-c6f4-46ba-830e-2a85e07a32c9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.496293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388121f-aa73-43a0-b61b-6340469e5193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f441c0592cd146d3be851ac96cbc49dc, user_id=89c747f18b434e469ae61755e9f639ca, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.497349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WARNING py.warnings [None req-0388121f-aa73-43a0-b61b-6340469e5193 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:51:45.497349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: warnings.warn(msg) Oct 03 14:51:45.497349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: Oct 03 14:51:45.499310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388121f-aa73-43a0-b61b-6340469e5193 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.513389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 791/1575] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:45 2022] PUT /identity/v3/projects/f441c0592cd146d3be851ac96cbc49dc/users/89c747f18b434e469ae61755e9f639ca/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:45.514355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388121f-aa73-43a0-b61b-6340469e5193 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.514489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 785/1576] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.515195 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dafe3-c6f4-46ba-830e-2a85e07a32c9 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.567022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:45.572993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:45.577821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388121f-aa73-43a0-b61b-6340469e5193 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.579666 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dafe3-c6f4-46ba-830e-2a85e07a32c9 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.582741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d98eb1-6d5f-40f6-8e42-18c400516cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.583396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d98eb1-6d5f-40f6-8e42-18c400516cff admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.583678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d98eb1-6d5f-40f6-8e42-18c400516cff admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.584076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d98eb1-6d5f-40f6-8e42-18c400516cff admin admin] PATH_INFO: `/v3/projects/83b495f3cb9b437d9572f4d33327fbf7/users/335dd8706d4c402e98605479b57d059a/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.584957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93d481-c244-45fa-87f7-12d42166f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.585579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93d481-c244-45fa-87f7-12d42166f772 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.585859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93d481-c244-45fa-87f7-12d42166f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.586129 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93d481-c244-45fa-87f7-12d42166f772 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.586761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93d481-c244-45fa-87f7-12d42166f772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.588668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93d481-c244-45fa-87f7-12d42166f772 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.602279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d98eb1-6d5f-40f6-8e42-18c400516cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b495f3cb9b437d9572f4d33327fbf7, user_id=335dd8706d4c402e98605479b57d059a, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.604949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d98eb1-6d5f-40f6-8e42-18c400516cff admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.617423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 786/1577] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.618484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93d481-c244-45fa-87f7-12d42166f772 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.618988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 792/1578] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:45 2022] PUT /identity/v3/projects/83b495f3cb9b437d9572f4d33327fbf7/users/335dd8706d4c402e98605479b57d059a/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:45.619857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d98eb1-6d5f-40f6-8e42-18c400516cff admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.682677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d98eb1-6d5f-40f6-8e42-18c400516cff admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.699557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93d481-c244-45fa-87f7-12d42166f772 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.704216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562e7cc-a0cb-42eb-bf62-401f4ff5284f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.704825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562e7cc-a0cb-42eb-bf62-401f4ff5284f admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.705106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562e7cc-a0cb-42eb-bf62-401f4ff5284f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.705381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562e7cc-a0cb-42eb-bf62-401f4ff5284f admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.706049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562e7cc-a0cb-42eb-bf62-401f4ff5284f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.708050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562e7cc-a0cb-42eb-bf62-401f4ff5284f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.730272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 787/1579] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.731075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562e7cc-a0cb-42eb-bf62-401f4ff5284f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.739941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957aeb0-3e03-469a-9110-81c258f9683c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.741012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562e7cc-a0cb-42eb-bf62-401f4ff5284f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.741134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957aeb0-3e03-469a-9110-81c258f9683c admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.741376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957aeb0-3e03-469a-9110-81c258f9683c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.741676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957aeb0-3e03-469a-9110-81c258f9683c admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.742333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957aeb0-3e03-469a-9110-81c258f9683c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.744276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957aeb0-3e03-469a-9110-81c258f9683c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.746225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66391b7a-0e00-4280-8b6c-bb500a2f401f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.746940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66391b7a-0e00-4280-8b6c-bb500a2f401f admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.747220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66391b7a-0e00-4280-8b6c-bb500a2f401f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.747491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66391b7a-0e00-4280-8b6c-bb500a2f401f admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.748109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66391b7a-0e00-4280-8b6c-bb500a2f401f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.750559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66391b7a-0e00-4280-8b6c-bb500a2f401f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.771716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 793/1580] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.772785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957aeb0-3e03-469a-9110-81c258f9683c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.775825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 788/1581] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.776648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66391b7a-0e00-4280-8b6c-bb500a2f401f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.786810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66391b7a-0e00-4280-8b6c-bb500a2f401f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.836185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957aeb0-3e03-469a-9110-81c258f9683c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.840852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0727f-6b4b-4981-ad2a-2d1d0d2f1cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.841455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0727f-6b4b-4981-ad2a-2d1d0d2f1cea admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.841763 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0727f-6b4b-4981-ad2a-2d1d0d2f1cea admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.842034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0727f-6b4b-4981-ad2a-2d1d0d2f1cea admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.842160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef24ea0-a34c-4601-80f9-458f55c4c4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.842684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0727f-6b4b-4981-ad2a-2d1d0d2f1cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.842800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24ea0-a34c-4601-80f9-458f55c4c4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.842945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24ea0-a34c-4601-80f9-458f55c4c4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.843215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24ea0-a34c-4601-80f9-458f55c4c4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.843831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef24ea0-a34c-4601-80f9-458f55c4c4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.844841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0727f-6b4b-4981-ad2a-2d1d0d2f1cea admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.846113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef24ea0-a34c-4601-80f9-458f55c4c4a3 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.862323 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:45.872903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 789/1582] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.872903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c5521-2721-4521-83ca-534972e27d9e None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.872903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c5521-2721-4521-83ca-534972e27d9e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.872903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c5521-2721-4521-83ca-534972e27d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.873626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 794/1583] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.874579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0727f-6b4b-4981-ad2a-2d1d0d2f1cea admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.884965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0727f-6b4b-4981-ad2a-2d1d0d2f1cea admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.889692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87e80c-22c4-4056-bd0a-c85facf0eaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.890280 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87e80c-22c4-4056-bd0a-c85facf0eaeb admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.890561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87e80c-22c4-4056-bd0a-c85facf0eaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.890839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87e80c-22c4-4056-bd0a-c85facf0eaeb admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.890952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e68c5521-2721-4521-83ca-534972e27d9e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:45.891458 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87e80c-22c4-4056-bd0a-c85facf0eaeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.892171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e68c5521-2721-4521-83ca-534972e27d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:45.893377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87e80c-22c4-4056-bd0a-c85facf0eaeb admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.900291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 790/1584] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:45.901435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729796e-afdb-4ea1-ba0e-4ce471c6f395 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.901746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729796e-afdb-4ea1-ba0e-4ce471c6f395 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.902016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729796e-afdb-4ea1-ba0e-4ce471c6f395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.916511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 795/1585] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.917237 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87e80c-22c4-4056-bd0a-c85facf0eaeb admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.921755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d729796e-afdb-4ea1-ba0e-4ce471c6f395 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:45.922977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d729796e-afdb-4ea1-ba0e-4ce471c6f395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:45.926973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87e80c-22c4-4056-bd0a-c85facf0eaeb admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.930826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 791/1586] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:45.931577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729796e-afdb-4ea1-ba0e-4ce471c6f395 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.931697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a829ca-e360-491b-b6f2-46f10fb058c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.932172 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a829ca-e360-491b-b6f2-46f10fb058c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.932433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a829ca-e360-491b-b6f2-46f10fb058c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.932699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a829ca-e360-491b-b6f2-46f10fb058c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.933333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a829ca-e360-491b-b6f2-46f10fb058c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.934955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a829ca-e360-491b-b6f2-46f10fb058c8 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.941275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729796e-afdb-4ea1-ba0e-4ce471c6f395 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.948437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 796/1587] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:45.948590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3b308-368d-4ff1-9782-826f97dd8bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.948734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a829ca-e360-491b-b6f2-46f10fb058c8 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.948839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3b308-368d-4ff1-9782-826f97dd8bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.948839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3b308-368d-4ff1-9782-826f97dd8bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.948839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3b308-368d-4ff1-9782-826f97dd8bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.948839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3b308-368d-4ff1-9782-826f97dd8bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.949812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3b308-368d-4ff1-9782-826f97dd8bc3 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.957792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a829ca-e360-491b-b6f2-46f10fb058c8 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.963151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd939bf-42be-452a-81b7-68b85cb55e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.963940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd939bf-42be-452a-81b7-68b85cb55e37 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.964300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd939bf-42be-452a-81b7-68b85cb55e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.964672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd939bf-42be-452a-81b7-68b85cb55e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.965470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd939bf-42be-452a-81b7-68b85cb55e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.967222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd939bf-42be-452a-81b7-68b85cb55e37 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.972524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 792/1588] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:45 2022] POST /identity/v3/users => generated 283 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:45.973335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3b308-368d-4ff1-9782-826f97dd8bc3 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.981194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 797/1589] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:45.981194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd939bf-42be-452a-81b7-68b85cb55e37 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:45.984291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3b308-368d-4ff1-9782-826f97dd8bc3 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.988213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd939bf-42be-452a-81b7-68b85cb55e37 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:45.989020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347f0fa-e243-4c70-b3cf-b8dfd1d6fefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.989652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347f0fa-e243-4c70-b3cf-b8dfd1d6fefd admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.989934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347f0fa-e243-4c70-b3cf-b8dfd1d6fefd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.990243 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347f0fa-e243-4c70-b3cf-b8dfd1d6fefd admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.990886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347f0fa-e243-4c70-b3cf-b8dfd1d6fefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.992493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347f0fa-e243-4c70-b3cf-b8dfd1d6fefd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.993455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fb8e1-e534-4e79-adde-af1a34cdb6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:45.994368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fb8e1-e534-4e79-adde-af1a34cdb6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:45.995043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fb8e1-e534-4e79-adde-af1a34cdb6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:45.995321 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fb8e1-e534-4e79-adde-af1a34cdb6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:45.995963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fb8e1-e534-4e79-adde-af1a34cdb6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:45.997547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fb8e1-e534-4e79-adde-af1a34cdb6d7 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:45.997906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:46.004359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 793/1590] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:46.006069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347f0fa-e243-4c70-b3cf-b8dfd1d6fefd admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.008925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 798/1591] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:46.010414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fb8e1-e534-4e79-adde-af1a34cdb6d7 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.020521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fb8e1-e534-4e79-adde-af1a34cdb6d7 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.025656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90cd09-77af-4874-bf5d-fcd3cbd8c444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.026337 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90cd09-77af-4874-bf5d-fcd3cbd8c444 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.026579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90cd09-77af-4874-bf5d-fcd3cbd8c444 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.026875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90cd09-77af-4874-bf5d-fcd3cbd8c444 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.027499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90cd09-77af-4874-bf5d-fcd3cbd8c444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.029119 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90cd09-77af-4874-bf5d-fcd3cbd8c444 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.040229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 799/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:46.040840 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90cd09-77af-4874-bf5d-fcd3cbd8c444 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.042531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-0f90cd09-77af-4874-bf5d-fcd3cbd8c444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:46.064615 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347f0fa-e243-4c70-b3cf-b8dfd1d6fefd admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.068289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab516850-e8dd-49f2-80e4-643782a82840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.068957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab516850-e8dd-49f2-80e4-643782a82840 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.069239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab516850-e8dd-49f2-80e4-643782a82840 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.069533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab516850-e8dd-49f2-80e4-643782a82840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.079367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab516850-e8dd-49f2-80e4-643782a82840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.080804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab516850-e8dd-49f2-80e4-643782a82840 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.091040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 794/1593] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:46.091780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab516850-e8dd-49f2-80e4-643782a82840 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.100960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90cd09-77af-4874-bf5d-fcd3cbd8c444 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.101450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab516850-e8dd-49f2-80e4-643782a82840 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.104883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49840f-7815-4a29-83ce-34d852729fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.105748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49840f-7815-4a29-83ce-34d852729fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.105868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49840f-7815-4a29-83ce-34d852729fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.106063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49840f-7815-4a29-83ce-34d852729fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.106595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5af5d0-6a82-429b-8349-0e319b6fdf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.107271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5af5d0-6a82-429b-8349-0e319b6fdf30 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.108187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5af5d0-6a82-429b-8349-0e319b6fdf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.108187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5af5d0-6a82-429b-8349-0e319b6fdf30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.108773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5af5d0-6a82-429b-8349-0e319b6fdf30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.110489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5af5d0-6a82-429b-8349-0e319b6fdf30 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.116225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49840f-7815-4a29-83ce-34d852729fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.117903 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49840f-7815-4a29-83ce-34d852729fa0 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.121719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 795/1594] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:46.122657 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5af5d0-6a82-429b-8349-0e319b6fdf30 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.128645 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 800/1595] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:46.129431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49840f-7815-4a29-83ce-34d852729fa0 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.133014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5af5d0-6a82-429b-8349-0e319b6fdf30 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.137874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe1880-ea0c-40aa-a82f-adf2b4551d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.138526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe1880-ea0c-40aa-a82f-adf2b4551d56 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.138813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe1880-ea0c-40aa-a82f-adf2b4551d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.139098 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe1880-ea0c-40aa-a82f-adf2b4551d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.139500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49840f-7815-4a29-83ce-34d852729fa0 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.139749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe1880-ea0c-40aa-a82f-adf2b4551d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.141538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe1880-ea0c-40aa-a82f-adf2b4551d56 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.151811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bd86b-cb5a-41e7-80f0-9f9b5bef36fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.152478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bd86b-cb5a-41e7-80f0-9f9b5bef36fa admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.152801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bd86b-cb5a-41e7-80f0-9f9b5bef36fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.153078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bd86b-cb5a-41e7-80f0-9f9b5bef36fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.153960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bd86b-cb5a-41e7-80f0-9f9b5bef36fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.155713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bd86b-cb5a-41e7-80f0-9f9b5bef36fa admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.158212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 796/1596] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:46.159508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe1880-ea0c-40aa-a82f-adf2b4551d56 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.172258 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 801/1597] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:46.173222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bd86b-cb5a-41e7-80f0-9f9b5bef36fa admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.175791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe1880-ea0c-40aa-a82f-adf2b4551d56 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.183377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621143ae-bea4-4e93-a30b-1a23169b4857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.184543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bd86b-cb5a-41e7-80f0-9f9b5bef36fa admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.184753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621143ae-bea4-4e93-a30b-1a23169b4857 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.185186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621143ae-bea4-4e93-a30b-1a23169b4857 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.186011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621143ae-bea4-4e93-a30b-1a23169b4857 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.187092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621143ae-bea4-4e93-a30b-1a23169b4857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.190454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621143ae-bea4-4e93-a30b-1a23169b4857 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.190636 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280059f-dd16-4d15-826d-d2744f2dfa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.191300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280059f-dd16-4d15-826d-d2744f2dfa58 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.191586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280059f-dd16-4d15-826d-d2744f2dfa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.191946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280059f-dd16-4d15-826d-d2744f2dfa58 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.192637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280059f-dd16-4d15-826d-d2744f2dfa58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.194839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280059f-dd16-4d15-826d-d2744f2dfa58 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.207371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 797/1598] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:46.209516 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621143ae-bea4-4e93-a30b-1a23169b4857 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.212029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 802/1599] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:46.213015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280059f-dd16-4d15-826d-d2744f2dfa58 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.224267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280059f-dd16-4d15-826d-d2744f2dfa58 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.226063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621143ae-bea4-4e93-a30b-1a23169b4857 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.230799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93984fd-030b-4bcf-871b-0f5e5b01ee0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.231529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93984fd-030b-4bcf-871b-0f5e5b01ee0a admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.231791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93984fd-030b-4bcf-871b-0f5e5b01ee0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.232219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93984fd-030b-4bcf-871b-0f5e5b01ee0a admin admin] PATH_INFO: `/v3/projects/b27a281133464bd5840c6a8b40e251f9/users/b757dbe5fa0b4465b93d672981b84b80/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.233619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58c05e-faf2-49cb-95a0-83c62bd6b10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.234515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58c05e-faf2-49cb-95a0-83c62bd6b10a admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.235372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58c05e-faf2-49cb-95a0-83c62bd6b10a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.236717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58c05e-faf2-49cb-95a0-83c62bd6b10a admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.237648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58c05e-faf2-49cb-95a0-83c62bd6b10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.240198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58c05e-faf2-49cb-95a0-83c62bd6b10a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.251762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93984fd-030b-4bcf-871b-0f5e5b01ee0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27a281133464bd5840c6a8b40e251f9, user_id=b757dbe5fa0b4465b93d672981b84b80, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.254917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 798/1600] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:46.256729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58c05e-faf2-49cb-95a0-83c62bd6b10a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.260150 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93984fd-030b-4bcf-871b-0f5e5b01ee0a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.268473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58c05e-faf2-49cb-95a0-83c62bd6b10a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.276228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3acfe1-7b8d-489e-9578-ae79495c4f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.277481 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3acfe1-7b8d-489e-9578-ae79495c4f58 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.277787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3acfe1-7b8d-489e-9578-ae79495c4f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.278060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3acfe1-7b8d-489e-9578-ae79495c4f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.289245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 803/1601] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:46 2022] PUT /identity/v3/projects/b27a281133464bd5840c6a8b40e251f9/users/b757dbe5fa0b4465b93d672981b84b80/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:46.289489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3acfe1-7b8d-489e-9578-ae79495c4f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.289489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3acfe1-7b8d-489e-9578-ae79495c4f58 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.289701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93984fd-030b-4bcf-871b-0f5e5b01ee0a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.294828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 799/1602] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:46.295695 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3acfe1-7b8d-489e-9578-ae79495c4f58 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.308184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3acfe1-7b8d-489e-9578-ae79495c4f58 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.363724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:46.383821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93984fd-030b-4bcf-871b-0f5e5b01ee0a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.386686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a82668-484b-46ee-9257-6b4d0081f112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.387572 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a82668-484b-46ee-9257-6b4d0081f112 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.388582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a82668-484b-46ee-9257-6b4d0081f112 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.388931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a82668-484b-46ee-9257-6b4d0081f112 admin admin] PATH_INFO: `/v3/projects/7f48b2441eec48c4ac784844e475a9e9/users/9e5325f010ae4359bd6b514a1995614d/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.390517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7470680-78ed-426b-b4de-9c885764a129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.391448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7470680-78ed-426b-b4de-9c885764a129 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.391952 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7470680-78ed-426b-b4de-9c885764a129 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.392409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7470680-78ed-426b-b4de-9c885764a129 admin admin] PATH_INFO: `/v3/projects/8cf070b7d6ff419392acaddbbee1e242/users/0be78e1abb3947a2ac5881b9fd2f6fae/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.423863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7470680-78ed-426b-b4de-9c885764a129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf070b7d6ff419392acaddbbee1e242, user_id=0be78e1abb3947a2ac5881b9fd2f6fae, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.427634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7470680-78ed-426b-b4de-9c885764a129 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.439545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a82668-484b-46ee-9257-6b4d0081f112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f48b2441eec48c4ac784844e475a9e9, user_id=9e5325f010ae4359bd6b514a1995614d, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.442952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a82668-484b-46ee-9257-6b4d0081f112 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.454714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 804/1603] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:46 2022] PUT /identity/v3/projects/8cf070b7d6ff419392acaddbbee1e242/users/0be78e1abb3947a2ac5881b9fd2f6fae/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:46.455173 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7470680-78ed-426b-b4de-9c885764a129 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.466894 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 800/1604] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:46 2022] PUT /identity/v3/projects/7f48b2441eec48c4ac784844e475a9e9/users/9e5325f010ae4359bd6b514a1995614d/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:46.468081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a82668-484b-46ee-9257-6b4d0081f112 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.507734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:46.568157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a82668-484b-46ee-9257-6b4d0081f112 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.574087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71794f-d2bc-4b3c-8ff4-70c967bc729f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.574911 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71794f-d2bc-4b3c-8ff4-70c967bc729f admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.575035 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7470680-78ed-426b-b4de-9c885764a129 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.575421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71794f-d2bc-4b3c-8ff4-70c967bc729f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.575907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71794f-d2bc-4b3c-8ff4-70c967bc729f admin admin] PATH_INFO: `/v3/projects/85e2a5410c7f43b88a9b4c643b80ae85/users/187717bf784d4526a949a13d0e437237/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.586096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c32ec-e48e-4215-96be-7ff9edbe2c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.587036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c32ec-e48e-4215-96be-7ff9edbe2c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.587321 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c32ec-e48e-4215-96be-7ff9edbe2c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.587597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c32ec-e48e-4215-96be-7ff9edbe2c99 admin admin] PATH_INFO: `/v3/projects/2e3b8c8420474d9397a3578350b9eacf/users/913c969eb6274750aef880400a74e637/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.612776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71794f-d2bc-4b3c-8ff4-70c967bc729f admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e2a5410c7f43b88a9b4c643b80ae85, user_id=187717bf784d4526a949a13d0e437237, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.614425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c32ec-e48e-4215-96be-7ff9edbe2c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3b8c8420474d9397a3578350b9eacf, user_id=913c969eb6274750aef880400a74e637, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.616745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71794f-d2bc-4b3c-8ff4-70c967bc729f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.618179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c32ec-e48e-4215-96be-7ff9edbe2c99 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.635310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 801/1605] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:46 2022] PUT /identity/v3/projects/85e2a5410c7f43b88a9b4c643b80ae85/users/187717bf784d4526a949a13d0e437237/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:46.636930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db2a03-85a1-44ca-b4cd-a281d54040c3 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.637358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db2a03-85a1-44ca-b4cd-a281d54040c3 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.637807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db2a03-85a1-44ca-b4cd-a281d54040c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.643794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 805/1606] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:46 2022] PUT /identity/v3/projects/2e3b8c8420474d9397a3578350b9eacf/users/913c969eb6274750aef880400a74e637/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:46.645026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c32ec-e48e-4215-96be-7ff9edbe2c99 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.678573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-55db2a03-85a1-44ca-b4cd-a281d54040c3 None None] MFA Rules not processed for user `b757dbe5fa0b4465b93d672981b84b80`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:46.680899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-55db2a03-85a1-44ca-b4cd-a281d54040c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:46.748714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c32ec-e48e-4215-96be-7ff9edbe2c99 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.751853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a68744-9f88-4b4c-807c-0c9b5fe87aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.752023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a68744-9f88-4b4c-807c-0c9b5fe87aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.752239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a68744-9f88-4b4c-807c-0c9b5fe87aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.752511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a68744-9f88-4b4c-807c-0c9b5fe87aaa admin admin] PATH_INFO: `/v3/projects/56485c17468e448cb52e2c2206e8c6c2/users/33088e0787eb4ed5ba780f036258c082/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.779115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a68744-9f88-4b4c-807c-0c9b5fe87aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=56485c17468e448cb52e2c2206e8c6c2, user_id=33088e0787eb4ed5ba780f036258c082, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.782466 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a68744-9f88-4b4c-807c-0c9b5fe87aaa admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:46.804519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 802/1607] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:46.807082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 806/1608] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:46 2022] PUT /identity/v3/projects/56485c17468e448cb52e2c2206e8c6c2/users/33088e0787eb4ed5ba780f036258c082/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:46.813553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3442-a940-4951-9188-cde171e3c6a1 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.813553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3442-a940-4951-9188-cde171e3c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.813553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3442-a940-4951-9188-cde171e3c6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.814392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed104491-3d06-407b-9963-6905d1e36930 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.815984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed104491-3d06-407b-9963-6905d1e36930 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.817424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed104491-3d06-407b-9963-6905d1e36930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.839061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-9e8e3442-a940-4951-9188-cde171e3c6a1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:46.840681 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9e8e3442-a940-4951-9188-cde171e3c6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:46.858976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ed104491-3d06-407b-9963-6905d1e36930 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:46.860176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ed104491-3d06-407b-9963-6905d1e36930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:46.890389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:46.921732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 807/1609] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:46.924128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78db0-dc02-4238-9ddf-d7869442c129 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.925234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78db0-dc02-4238-9ddf-d7869442c129 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.925720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78db0-dc02-4238-9ddf-d7869442c129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.928603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 803/1610] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:46.930250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a234-f1ca-4fb9-8159-1454475275f1 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.930613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a234-f1ca-4fb9-8159-1454475275f1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.930963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a234-f1ca-4fb9-8159-1454475275f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.952936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5de78db0-dc02-4238-9ddf-d7869442c129 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:46.954521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5de78db0-dc02-4238-9ddf-d7869442c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:46.964553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 808/1611] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:46.964857 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6076a234-f1ca-4fb9-8159-1454475275f1 None None] MFA Rules not processed for user `187717bf784d4526a949a13d0e437237`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:46.966100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6076a234-f1ca-4fb9-8159-1454475275f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:46.966658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78db0-dc02-4238-9ddf-d7869442c129 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:46.980708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78db0-dc02-4238-9ddf-d7869442c129 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:46.987308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be814178-aca6-40f0-92e0-c6e7263fd6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:46.988125 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be814178-aca6-40f0-92e0-c6e7263fd6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:46.989093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be814178-aca6-40f0-92e0-c6e7263fd6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:46.989093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be814178-aca6-40f0-92e0-c6e7263fd6b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:46.989592 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be814178-aca6-40f0-92e0-c6e7263fd6b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:46.992883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be814178-aca6-40f0-92e0-c6e7263fd6b9 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.007141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:47.028106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 809/1612] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:46 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:47.029701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97696b5f-482d-4113-8d1d-5ffc622cf77c None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.030140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97696b5f-482d-4113-8d1d-5ffc622cf77c None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.030601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97696b5f-482d-4113-8d1d-5ffc622cf77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.046837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 804/1613] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:47.047870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a234-f1ca-4fb9-8159-1454475275f1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:47.071496 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-97696b5f-482d-4113-8d1d-5ffc622cf77c None None] MFA Rules not processed for user `33088e0787eb4ed5ba780f036258c082`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:47.073629 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-97696b5f-482d-4113-8d1d-5ffc622cf77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:47.130419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a234-f1ca-4fb9-8159-1454475275f1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:47.171390 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 810/1614] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:47.171390 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97696b5f-482d-4113-8d1d-5ffc622cf77c None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:47.206558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6af89-812f-486e-be79-02a64276404f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:47.207188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6af89-812f-486e-be79-02a64276404f service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.207465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6af89-812f-486e-be79-02a64276404f service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.207742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6af89-812f-486e-be79-02a64276404f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.242773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97696b5f-482d-4113-8d1d-5ffc622cf77c None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:47.248170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed245133-59b4-4ea2-b3c8-23401bf0f4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:47.248858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed245133-59b4-4ea2-b3c8-23401bf0f4aa service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.249134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed245133-59b4-4ea2-b3c8-23401bf0f4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.249522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed245133-59b4-4ea2-b3c8-23401bf0f4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.315530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6af89-812f-486e-be79-02a64276404f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:47.317256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6af89-812f-486e-be79-02a64276404f service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.332775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 805/1615] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:47.334472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6af89-812f-486e-be79-02a64276404f service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:47.346256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6af89-812f-486e-be79-02a64276404f service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:47.350310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:47.351114 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddacb42-a5c6-40a9-8e29-24c79862399f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:47.351939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddacb42-a5c6-40a9-8e29-24c79862399f service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.352199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddacb42-a5c6-40a9-8e29-24c79862399f service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.352445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddacb42-a5c6-40a9-8e29-24c79862399f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.359778 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed245133-59b4-4ea2-b3c8-23401bf0f4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:47.362865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed245133-59b4-4ea2-b3c8-23401bf0f4aa service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.366872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddacb42-a5c6-40a9-8e29-24c79862399f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:47.368786 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddacb42-a5c6-40a9-8e29-24c79862399f service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.383397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 806/1616] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:47.384395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddacb42-a5c6-40a9-8e29-24c79862399f service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:47.384856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 811/1617] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:47.386595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42324773-aab0-46dd-8905-1bf3cf949012 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.387077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42324773-aab0-46dd-8905-1bf3cf949012 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.387528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42324773-aab0-46dd-8905-1bf3cf949012 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.389156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 812/1618] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 3 14:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:51:47.390525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42324773-aab0-46dd-8905-1bf3cf949012 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:47.395475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddacb42-a5c6-40a9-8e29-24c79862399f service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:47.401681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dfc77-5df9-4c79-9984-ad3bdef04a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:47.408151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dfc77-5df9-4c79-9984-ad3bdef04a25 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.414197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dfc77-5df9-4c79-9984-ad3bdef04a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.414593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dfc77-5df9-4c79-9984-ad3bdef04a25 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.415440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dfc77-5df9-4c79-9984-ad3bdef04a25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:47.418040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dfc77-5df9-4c79-9984-ad3bdef04a25 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.458310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 807/1619] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:47 2022] POST /identity/v3/users => generated 297 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:47.458310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1b8dc-aee3-4dfe-a1f5-e8a04a96d9e6 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.459018 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1b8dc-aee3-4dfe-a1f5-e8a04a96d9e6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.459411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1b8dc-aee3-4dfe-a1f5-e8a04a96d9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.475035 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42324773-aab0-46dd-8905-1bf3cf949012 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:47.482852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffc21b-972c-4c28-98c2-debf8dde401b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:47.483657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc21b-972c-4c28-98c2-debf8dde401b service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.484113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc21b-972c-4c28-98c2-debf8dde401b service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.484678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc21b-972c-4c28-98c2-debf8dde401b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.486987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:47.498238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffc21b-972c-4c28-98c2-debf8dde401b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:47.500116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffc21b-972c-4c28-98c2-debf8dde401b service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.515304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 813/1620] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:47.516336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffc21b-972c-4c28-98c2-debf8dde401b service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:47.516655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-97b1b8dc-aee3-4dfe-a1f5-e8a04a96d9e6 None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:47.518147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-97b1b8dc-aee3-4dfe-a1f5-e8a04a96d9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:47.530619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffc21b-972c-4c28-98c2-debf8dde401b service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:47.539805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdd4dd-718f-4aac-b3a9-80392de86a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:47.540743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdd4dd-718f-4aac-b3a9-80392de86a9a admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.541132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdd4dd-718f-4aac-b3a9-80392de86a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.541511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdd4dd-718f-4aac-b3a9-80392de86a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.542456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdd4dd-718f-4aac-b3a9-80392de86a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:47.544975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdd4dd-718f-4aac-b3a9-80392de86a9a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.563148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 814/1621] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:47.567193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdd4dd-718f-4aac-b3a9-80392de86a9a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:47.581969 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdd4dd-718f-4aac-b3a9-80392de86a9a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:47.590713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ca8aa-61c4-4f3f-af14-a03b99883d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:47.591933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ca8aa-61c4-4f3f-af14-a03b99883d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.592409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ca8aa-61c4-4f3f-af14-a03b99883d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.592850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ca8aa-61c4-4f3f-af14-a03b99883d0f admin admin] PATH_INFO: `/v3/projects/a1261ce2be2f46749fe7ffb31192999a/users/dea901f3bc9f40859d71d3d5b40d067c/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.613793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ca8aa-61c4-4f3f-af14-a03b99883d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1261ce2be2f46749fe7ffb31192999a, user_id=dea901f3bc9f40859d71d3d5b40d067c, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:47.619104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ca8aa-61c4-4f3f-af14-a03b99883d0f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.628576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 808/1622] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 3 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:47.639264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1b8dc-aee3-4dfe-a1f5-e8a04a96d9e6 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:47.646164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 815/1623] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:47 2022] PUT /identity/v3/projects/a1261ce2be2f46749fe7ffb31192999a/users/dea901f3bc9f40859d71d3d5b40d067c/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:47.648800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ca8aa-61c4-4f3f-af14-a03b99883d0f admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:47.652149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1b8dc-aee3-4dfe-a1f5-e8a04a96d9e6 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:47.744301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a218a2-b1db-47da-a425-58b5d309c565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:47.745168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a218a2-b1db-47da-a425-58b5d309c565 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.745626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a218a2-b1db-47da-a425-58b5d309c565 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.747149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a218a2-b1db-47da-a425-58b5d309c565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.811196 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ca8aa-61c4-4f3f-af14-a03b99883d0f admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:47.828004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f55e1-8b80-4b38-92ef-ff16d4273cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:47.829033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f55e1-8b80-4b38-92ef-ff16d4273cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.829420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f55e1-8b80-4b38-92ef-ff16d4273cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.829809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f55e1-8b80-4b38-92ef-ff16d4273cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.830578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f55e1-8b80-4b38-92ef-ff16d4273cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:47.833312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f55e1-8b80-4b38-92ef-ff16d4273cf0 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.867961 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 816/1624] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:47.873721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a218a2-b1db-47da-a425-58b5d309c565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:47.873962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f55e1-8b80-4b38-92ef-ff16d4273cf0 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:47.875363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a218a2-b1db-47da-a425-58b5d309c565 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.893633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 809/1625] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:47.909168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f55e1-8b80-4b38-92ef-ff16d4273cf0 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:47.934514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8801e-8953-49b9-a244-63bd34693a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:47.935317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8801e-8953-49b9-a244-63bd34693a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.935671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8801e-8953-49b9-a244-63bd34693a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.936475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8801e-8953-49b9-a244-63bd34693a5f admin admin] PATH_INFO: `/v3/projects/a1261ce2be2f46749fe7ffb31192999a/users/dea901f3bc9f40859d71d3d5b40d067c/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:47.942157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8801e-8953-49b9-a244-63bd34693a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1261ce2be2f46749fe7ffb31192999a, user_id=dea901f3bc9f40859d71d3d5b40d067c, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:47.951150 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8801e-8953-49b9-a244-63bd34693a5f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:47.983079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 817/1626] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:47 2022] PUT /identity/v3/projects/a1261ce2be2f46749fe7ffb31192999a/users/dea901f3bc9f40859d71d3d5b40d067c/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:47.993908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9dd34-31b7-466b-aa25-9e57af2cc8da None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:47.994205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9dd34-31b7-466b-aa25-9e57af2cc8da None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:47.994757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9dd34-31b7-466b-aa25-9e57af2cc8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:48.051130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b3b9dd34-31b7-466b-aa25-9e57af2cc8da None None] MFA Rules not processed for user `dea901f3bc9f40859d71d3d5b40d067c`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:48.053149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b3b9dd34-31b7-466b-aa25-9e57af2cc8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:48.082137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:48.197776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 810/1627] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:48.207676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8801e-8953-49b9-a244-63bd34693a5f admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:48.356825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:48.391692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8801e-8953-49b9-a244-63bd34693a5f admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:48.404432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca89011-1f7c-4ab1-b682-18ea35133fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:48.407195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca89011-1f7c-4ab1-b682-18ea35133fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:48.423011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca89011-1f7c-4ab1-b682-18ea35133fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:48.423160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca89011-1f7c-4ab1-b682-18ea35133fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:48.450241 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca89011-1f7c-4ab1-b682-18ea35133fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:48.451778 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca89011-1f7c-4ab1-b682-18ea35133fd1 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:48.487516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 818/1628] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:49.003032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31346d9-e212-4f1b-a0ce-42bb439b89c7 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.003623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31346d9-e212-4f1b-a0ce-42bb439b89c7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.003623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31346d9-e212-4f1b-a0ce-42bb439b89c7 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.005530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 811/1629] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 3 14:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:51:49.020252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba3b35-4df3-44f9-a69b-60902d373e1f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.020760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba3b35-4df3-44f9-a69b-60902d373e1f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.022242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba3b35-4df3-44f9-a69b-60902d373e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.057259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-acba3b35-4df3-44f9-a69b-60902d373e1f None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:49.059270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-acba3b35-4df3-44f9-a69b-60902d373e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:49.204046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322db76d-3ce6-4c14-b49e-9b8b67ff94e5 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.204699 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322db76d-3ce6-4c14-b49e-9b8b67ff94e5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.204984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322db76d-3ce6-4c14-b49e-9b8b67ff94e5 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.206926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 812/1630] 10.222.0.46 () {54 vars in 912 bytes} [Mon Oct 3 14:51:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:51:49.216579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75226426-6439-4b59-b74d-5a9a23368dd2 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.216864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75226426-6439-4b59-b74d-5a9a23368dd2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.217143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75226426-6439-4b59-b74d-5a9a23368dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.230315 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 819/1631] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 3 14:51:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:49.264413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba3b35-4df3-44f9-a69b-60902d373e1f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:49.276379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-75226426-6439-4b59-b74d-5a9a23368dd2 None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:49.278134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-75226426-6439-4b59-b74d-5a9a23368dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:49.288472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba3b35-4df3-44f9-a69b-60902d373e1f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:49.302024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c77c17-475d-46e5-9d8b-43749f022056 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea901f3bc9f40859d71d3d5b40d067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1261ce2be2f46749fe7ffb31192999a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:49.302921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c77c17-475d-46e5-9d8b-43749f022056 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.303047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c77c17-475d-46e5-9d8b-43749f022056 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.303247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c77c17-475d-46e5-9d8b-43749f022056 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.303915 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c77c17-475d-46e5-9d8b-43749f022056 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:49.304222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c77c17-475d-46e5-9d8b-43749f022056 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:49.306333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c77c17-475d-46e5-9d8b-43749f022056 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:49.326941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 813/1632] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 14:51:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:51:49.336431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75226426-6439-4b59-b74d-5a9a23368dd2 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:49.365882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75226426-6439-4b59-b74d-5a9a23368dd2 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:49.376551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 820/1633] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:51:49.377719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a372ee-a999-4a58-80a9-4c1e86666ba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:49.378484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a372ee-a999-4a58-80a9-4c1e86666ba6 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.378773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a372ee-a999-4a58-80a9-4c1e86666ba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.379793 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a372ee-a999-4a58-80a9-4c1e86666ba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.382710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c77c17-475d-46e5-9d8b-43749f022056 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:49.404605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a372ee-a999-4a58-80a9-4c1e86666ba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:49.407081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c77c17-475d-46e5-9d8b-43749f022056 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:49.407367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a372ee-a999-4a58-80a9-4c1e86666ba6 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:49.418999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77fa64-94da-4851-a5e1-c0300b06ec5b tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea901f3bc9f40859d71d3d5b40d067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1261ce2be2f46749fe7ffb31192999a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:49.420877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77fa64-94da-4851-a5e1-c0300b06ec5b tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.421369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77fa64-94da-4851-a5e1-c0300b06ec5b tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.424822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77fa64-94da-4851-a5e1-c0300b06ec5b tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.425697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e77fa64-94da-4851-a5e1-c0300b06ec5b tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:49.427907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e77fa64-94da-4851-a5e1-c0300b06ec5b tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:49.429853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 814/1634] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:49.539031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 821/1635] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:49 2022] POST /identity/v3/projects => generated 354 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:49.544748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a372ee-a999-4a58-80a9-4c1e86666ba6 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:49.560213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e0430-05b8-4a34-a08a-819373320857 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.560449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a372ee-a999-4a58-80a9-4c1e86666ba6 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:49.560562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e0430-05b8-4a34-a08a-819373320857 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.560897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e0430-05b8-4a34-a08a-819373320857 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.562866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 822/1636] 10.222.0.46 () {54 vars in 912 bytes} [Mon Oct 3 14:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:51:49.568834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1abb07-ad16-4364-a9f5-acf4301f7462 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.570238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1abb07-ad16-4364-a9f5-acf4301f7462 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.570593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1abb07-ad16-4364-a9f5-acf4301f7462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.608718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-cc1abb07-ad16-4364-a9f5-acf4301f7462 None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:49.610146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-cc1abb07-ad16-4364-a9f5-acf4301f7462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:49.702080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c10f9-b255-4500-ad68-2c5633efe4b2 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea901f3bc9f40859d71d3d5b40d067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1261ce2be2f46749fe7ffb31192999a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:49.703073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c10f9-b255-4500-ad68-2c5633efe4b2 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.703487 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c10f9-b255-4500-ad68-2c5633efe4b2 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.703918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c10f9-b255-4500-ad68-2c5633efe4b2 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.705298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c10f9-b255-4500-ad68-2c5633efe4b2 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:49.705754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c10f9-b255-4500-ad68-2c5633efe4b2 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:49.709543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c10f9-b255-4500-ad68-2c5633efe4b2 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:49.743144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 815/1637] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:51:49.755457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c10f9-b255-4500-ad68-2c5633efe4b2 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:49.759136 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 823/1638] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 14:51:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:49.790909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1abb07-ad16-4364-a9f5-acf4301f7462 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:49.803232 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1abb07-ad16-4364-a9f5-acf4301f7462 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:49.809915 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea901f3bc9f40859d71d3d5b40d067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1261ce2be2f46749fe7ffb31192999a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:49.810813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:49.811259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:49.811684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] PATH_INFO: `/v3/projects/22321fc3bd834e9896931fac38b4e005` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:49.813002 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: Authorizing `identity:delete_project(project_id=22321fc3bd834e9896931fac38b4e005)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:49.815146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:49.865573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:49.888820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c10f9-b255-4500-ad68-2c5633efe4b2 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:49.895707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22321fc3bd834e9896931fac38b4e005', 'request_id': 'req-d1ed119c-15ed-44b2-941a-4b2073d503b7'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:49.911377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22321fc3bd834e9896931fac38b4e005', 'request_id': 'req-d1ed119c-15ed-44b2-941a-4b2073d503b7'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:49.933288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22321fc3bd834e9896931fac38b4e005', 'request_id': 'req-d1ed119c-15ed-44b2-941a-4b2073d503b7'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:49.952154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] The token cache is being invalidate because project 22321fc3bd834e9896931fac38b4e005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:51:49.952988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:49.954168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:49.957841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 824/1639] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:51:49 2022] DELETE /identity/v3/projects/22321fc3bd834e9896931fac38b4e005 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:51:49.977647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:50.008536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128255e9-82ca-4331-93ec-3bfc7037ae24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:50.009231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128255e9-82ca-4331-93ec-3bfc7037ae24 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:50.009601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128255e9-82ca-4331-93ec-3bfc7037ae24 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:50.009746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128255e9-82ca-4331-93ec-3bfc7037ae24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:50.013952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-128255e9-82ca-4331-93ec-3bfc7037ae24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:50.190391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128255e9-82ca-4331-93ec-3bfc7037ae24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:50.196535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128255e9-82ca-4331-93ec-3bfc7037ae24 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:50.241808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 816/1640] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:50.403580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:50.406519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:50.442611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128255e9-82ca-4331-93ec-3bfc7037ae24 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:50.444393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-128255e9-82ca-4331-93ec-3bfc7037ae24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:50.617948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed119c-15ed-44b2-941a-4b2073d503b7 tempest-VolumeQuotaClassesTest-2120396043 tempest-VolumeQuotaClassesTest-2120396043-project-admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:50.635113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e15750-d58e-4178-b805-390441662403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:50.635812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e15750-d58e-4178-b805-390441662403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:50.636442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e15750-d58e-4178-b805-390441662403 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:50.636540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e15750-d58e-4178-b805-390441662403 admin admin] PATH_INFO: `/v3/users/187717bf784d4526a949a13d0e437237` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:50.671578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e15750-d58e-4178-b805-390441662403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=187717bf784d4526a949a13d0e437237)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:50.674219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: WARNING py.warnings [None req-12e15750-d58e-4178-b805-390441662403 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:51:50.674219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: warnings.warn(msg) Oct 03 14:51:50.674219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: Oct 03 14:51:50.682796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e15750-d58e-4178-b805-390441662403 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:50.764419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128255e9-82ca-4331-93ec-3bfc7037ae24 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:50.783968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:50.786035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:50.786244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:50.788000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] PATH_INFO: `/v3/users/b757dbe5fa0b4465b93d672981b84b80` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:50.791321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b757dbe5fa0b4465b93d672981b84b80)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:50.793404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:50.826224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-12e15750-d58e-4178-b805-390441662403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187717bf784d4526a949a13d0e437237'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:50.829082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-12e15750-d58e-4178-b805-390441662403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187717bf784d4526a949a13d0e437237'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:50.844237 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-12e15750-d58e-4178-b805-390441662403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187717bf784d4526a949a13d0e437237'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:50.855906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-12e15750-d58e-4178-b805-390441662403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187717bf784d4526a949a13d0e437237'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:50.856888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-12e15750-d58e-4178-b805-390441662403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187717bf784d4526a949a13d0e437237'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:50.897895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b757dbe5fa0b4465b93d672981b84b80'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:50.898656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b757dbe5fa0b4465b93d672981b84b80'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:50.901056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 825/1641] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:51:50 2022] DELETE /identity/v3/users/187717bf784d4526a949a13d0e437237 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:51:50.907651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b44ea-bea2-4187-b1d1-9e98410696d1 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:50.908096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b44ea-bea2-4187-b1d1-9e98410696d1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:50.908507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b44ea-bea2-4187-b1d1-9e98410696d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:50.914774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b757dbe5fa0b4465b93d672981b84b80'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:50.935989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b757dbe5fa0b4465b93d672981b84b80'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:50.950450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f74b44ea-bea2-4187-b1d1-9e98410696d1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:50.952331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f74b44ea-bea2-4187-b1d1-9e98410696d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:50.955533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d7e39853-833d-43ca-b2ac-5c0edadaf353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b757dbe5fa0b4465b93d672981b84b80'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:50.969937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:50.975852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 817/1642] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:51:50 2022] DELETE /identity/v3/users/b757dbe5fa0b4465b93d672981b84b80 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:50.980174 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457b15f-321f-402f-8595-afbec427a793 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:50.980520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457b15f-321f-402f-8595-afbec427a793 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:50.980723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457b15f-321f-402f-8595-afbec427a793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:51.022627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:51.040377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2457b15f-321f-402f-8595-afbec427a793 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:51.046980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2457b15f-321f-402f-8595-afbec427a793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:51.139851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 826/1643] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:51.180908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 818/1644] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:51:51.210367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b44ea-bea2-4187-b1d1-9e98410696d1 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:51.211408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457b15f-321f-402f-8595-afbec427a793 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:51.212347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f74b44ea-bea2-4187-b1d1-9e98410696d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:51.212829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2457b15f-321f-402f-8595-afbec427a793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:51.335581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b44ea-bea2-4187-b1d1-9e98410696d1 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:51.398731 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457b15f-321f-402f-8595-afbec427a793 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:51.405517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da84ed-fe9a-40be-b865-5a311b2bf29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:51.406265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da84ed-fe9a-40be-b865-5a311b2bf29d service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:51.406552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da84ed-fe9a-40be-b865-5a311b2bf29d service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:51.407013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da84ed-fe9a-40be-b865-5a311b2bf29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:51.428833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da84ed-fe9a-40be-b865-5a311b2bf29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:51.431412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da84ed-fe9a-40be-b865-5a311b2bf29d service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:51.448919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c251e-c3a1-4437-8173-6606eca14b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:51.449642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c251e-c3a1-4437-8173-6606eca14b48 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:51.449998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c251e-c3a1-4437-8173-6606eca14b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:51.450340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c251e-c3a1-4437-8173-6606eca14b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:51.450981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 819/1645] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:51:51.473607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c251e-c3a1-4437-8173-6606eca14b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:51.474942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c251e-c3a1-4437-8173-6606eca14b48 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:51.515809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 827/1646] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:51:52.595898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da84ed-fe9a-40be-b865-5a311b2bf29d service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:52.600270 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-94da84ed-fe9a-40be-b865-5a311b2bf29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:52.656345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da84ed-fe9a-40be-b865-5a311b2bf29d service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:52.666435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:52.670505 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:52.670797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:52.671464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] PATH_INFO: `/v3/projects/b27a281133464bd5840c6a8b40e251f9` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:52.680526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27a281133464bd5840c6a8b40e251f9)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:52.682787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: WARNING py.warnings [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 14:51:52.682787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: warnings.warn(msg) Oct 03 14:51:52.682787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: Oct 03 14:51:52.684845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:52.805474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27a281133464bd5840c6a8b40e251f9', 'request_id': 'req-37e89c50-4cec-4906-846f-05089a822685'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:52.822412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27a281133464bd5840c6a8b40e251f9', 'request_id': 'req-37e89c50-4cec-4906-846f-05089a822685'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:52.872710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27a281133464bd5840c6a8b40e251f9', 'request_id': 'req-37e89c50-4cec-4906-846f-05089a822685'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:52.902057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] The token cache is being invalidate because project b27a281133464bd5840c6a8b40e251f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:51:52.902561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:52.904458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:52.907698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 820/1647] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:51:52 2022] DELETE /identity/v3/projects/b27a281133464bd5840c6a8b40e251f9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:52.911629 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c251e-c3a1-4437-8173-6606eca14b48 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:52.924422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c251e-c3a1-4437-8173-6606eca14b48 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:53.046247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:53.047465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:53.048026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:53.048659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] PATH_INFO: `/v3/users/dea901f3bc9f40859d71d3d5b40d067c` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:53.050161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea901f3bc9f40859d71d3d5b40d067c)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:53.053568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:53.126017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:53.136615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea901f3bc9f40859d71d3d5b40d067c'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:53.137757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea901f3bc9f40859d71d3d5b40d067c'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:53.157026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea901f3bc9f40859d71d3d5b40d067c'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:53.179780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea901f3bc9f40859d71d3d5b40d067c'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:53.179946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-346e2e8f-ad3f-4ccc-89a1-3c301f1923d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea901f3bc9f40859d71d3d5b40d067c'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:53.212203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 828/1648] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:51:52 2022] DELETE /identity/v3/users/dea901f3bc9f40859d71d3d5b40d067c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:53.434805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:53.437100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:53.464550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e89c50-4cec-4906-846f-05089a822685 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:53.570689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:53.707823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:53.709222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:53.709529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:53.709853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] PATH_INFO: `/v3/projects/85e2a5410c7f43b88a9b4c643b80ae85` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:53.712566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85e2a5410c7f43b88a9b4c643b80ae85)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:53.717273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:53.832148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85e2a5410c7f43b88a9b4c643b80ae85', 'request_id': 'req-66323fc4-c64e-4155-9a72-0be94f755bb3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:53.848840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85e2a5410c7f43b88a9b4c643b80ae85', 'request_id': 'req-66323fc4-c64e-4155-9a72-0be94f755bb3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:53.872553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85e2a5410c7f43b88a9b4c643b80ae85', 'request_id': 'req-66323fc4-c64e-4155-9a72-0be94f755bb3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:53.893480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] The token cache is being invalidate because project 85e2a5410c7f43b88a9b4c643b80ae85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:51:53.893821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:53.894507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66323fc4-c64e-4155-9a72-0be94f755bb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:53.899876 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 821/1649] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:51:53 2022] DELETE /identity/v3/projects/85e2a5410c7f43b88a9b4c643b80ae85 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:53.901551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17990e77-6da5-4fea-93cf-0f211b1e4398 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:53.901856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17990e77-6da5-4fea-93cf-0f211b1e4398 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:53.902141 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17990e77-6da5-4fea-93cf-0f211b1e4398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:53.945811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-17990e77-6da5-4fea-93cf-0f211b1e4398 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:53.950030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-17990e77-6da5-4fea-93cf-0f211b1e4398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:54.086688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 829/1650] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:54.088401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fb754-fb05-4104-85c7-62b5ad59ec91 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:54.088696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fb754-fb05-4104-85c7-62b5ad59ec91 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:54.088974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fb754-fb05-4104-85c7-62b5ad59ec91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:54.123898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-533fb754-fb05-4104-85c7-62b5ad59ec91 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:54.125135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-533fb754-fb05-4104-85c7-62b5ad59ec91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:54.134332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 822/1651] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:54.138270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17990e77-6da5-4fea-93cf-0f211b1e4398 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:54.151599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17990e77-6da5-4fea-93cf-0f211b1e4398 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:54.160981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f42ff8-a2b1-4b13-9144-94938269df60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:54.162092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f42ff8-a2b1-4b13-9144-94938269df60 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:54.163753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f42ff8-a2b1-4b13-9144-94938269df60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:54.164256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f42ff8-a2b1-4b13-9144-94938269df60 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:54.166267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f42ff8-a2b1-4b13-9144-94938269df60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:54.166802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f42ff8-a2b1-4b13-9144-94938269df60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:54.171758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f42ff8-a2b1-4b13-9144-94938269df60 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:54.199197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 830/1652] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:51:54.203366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fb754-fb05-4104-85c7-62b5ad59ec91 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:54.216932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fb754-fb05-4104-85c7-62b5ad59ec91 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:54.228651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b411c-c6fe-4b66-a9bf-7f2e65dfd017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:54.229312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b411c-c6fe-4b66-a9bf-7f2e65dfd017 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:54.229727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b411c-c6fe-4b66-a9bf-7f2e65dfd017 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:54.230040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b411c-c6fe-4b66-a9bf-7f2e65dfd017 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:54.230813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b411c-c6fe-4b66-a9bf-7f2e65dfd017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:54.232762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b411c-c6fe-4b66-a9bf-7f2e65dfd017 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:54.301597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 823/1653] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:54 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:54.304429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f42ff8-a2b1-4b13-9144-94938269df60 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:54.320069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f42ff8-a2b1-4b13-9144-94938269df60 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:54.484015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae20b3-6c89-46ab-9e46-d418964dc085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:54.485855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae20b3-6c89-46ab-9e46-d418964dc085 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:54.486300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae20b3-6c89-46ab-9e46-d418964dc085 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:54.486586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae20b3-6c89-46ab-9e46-d418964dc085 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:54.487276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae20b3-6c89-46ab-9e46-d418964dc085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:54.491934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae20b3-6c89-46ab-9e46-d418964dc085 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:54.540959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 831/1654] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:54 2022] POST /identity/v3/users => generated 286 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:51:54.542290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b411c-c6fe-4b66-a9bf-7f2e65dfd017 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:54.559714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b411c-c6fe-4b66-a9bf-7f2e65dfd017 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:54.577475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e35fae-96ba-42a1-9a6d-0b17154a2cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:54.578545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e35fae-96ba-42a1-9a6d-0b17154a2cdf admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:54.578659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e35fae-96ba-42a1-9a6d-0b17154a2cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:54.578909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e35fae-96ba-42a1-9a6d-0b17154a2cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:54.580527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e35fae-96ba-42a1-9a6d-0b17154a2cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:54.581451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e35fae-96ba-42a1-9a6d-0b17154a2cdf admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:54.608215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 824/1655] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:54.611994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae20b3-6c89-46ab-9e46-d418964dc085 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:54.643267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae20b3-6c89-46ab-9e46-d418964dc085 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:54.661310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162414e-9724-4973-8978-50cb971d2e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:54.662524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162414e-9724-4973-8978-50cb971d2e80 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:54.664439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162414e-9724-4973-8978-50cb971d2e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:54.664751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162414e-9724-4973-8978-50cb971d2e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:54.665431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162414e-9724-4973-8978-50cb971d2e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:54.667760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162414e-9724-4973-8978-50cb971d2e80 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:54.697475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 832/1656] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:54.701425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e35fae-96ba-42a1-9a6d-0b17154a2cdf admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:54.718598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e35fae-96ba-42a1-9a6d-0b17154a2cdf admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:54.727055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ec33f-4e6b-4f0e-9767-8d16a8e46e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:54.727953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ec33f-4e6b-4f0e-9767-8d16a8e46e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:54.728389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ec33f-4e6b-4f0e-9767-8d16a8e46e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:54.728906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ec33f-4e6b-4f0e-9767-8d16a8e46e7e admin admin] PATH_INFO: `/v3/projects/207889e387e740279c8a957c6b0b7362/users/8eb2d88cda4443429fb381ef1de08d7a/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:54.791201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ec33f-4e6b-4f0e-9767-8d16a8e46e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=207889e387e740279c8a957c6b0b7362, user_id=8eb2d88cda4443429fb381ef1de08d7a, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:54.799398 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ec33f-4e6b-4f0e-9767-8d16a8e46e7e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:54.829665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 825/1657] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:54 2022] PUT /identity/v3/projects/207889e387e740279c8a957c6b0b7362/users/8eb2d88cda4443429fb381ef1de08d7a/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:54.843435 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00cc5-a6ee-4262-afe7-e8d9434150f6 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:54.844947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00cc5-a6ee-4262-afe7-e8d9434150f6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:54.845278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00cc5-a6ee-4262-afe7-e8d9434150f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:54.932599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-3eb00cc5-a6ee-4262-afe7-e8d9434150f6 None None] MFA Rules not processed for user `8eb2d88cda4443429fb381ef1de08d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:54.934381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3eb00cc5-a6ee-4262-afe7-e8d9434150f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:55.010246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:55.258594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 833/1658] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:51:55.264075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ec33f-4e6b-4f0e-9767-8d16a8e46e7e admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:55.424758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:55.456000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ec33f-4e6b-4f0e-9767-8d16a8e46e7e admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:55.470213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa49fe-efbd-4d57-b0e0-75c0f0d637eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:55.473135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa49fe-efbd-4d57-b0e0-75c0f0d637eb admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:55.473642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa49fe-efbd-4d57-b0e0-75c0f0d637eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:55.475113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa49fe-efbd-4d57-b0e0-75c0f0d637eb admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:55.475113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa49fe-efbd-4d57-b0e0-75c0f0d637eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:55.476772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa49fe-efbd-4d57-b0e0-75c0f0d637eb admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:55.533554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 826/1659] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:55 2022] POST /identity/v3/projects => generated 371 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:55.536882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00cc5-a6ee-4262-afe7-e8d9434150f6 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:55.564147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00cc5-a6ee-4262-afe7-e8d9434150f6 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:55.718669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171dd82-e041-4ac5-a650-2695da2db57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:55.720327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171dd82-e041-4ac5-a650-2695da2db57f admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:55.720327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171dd82-e041-4ac5-a650-2695da2db57f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:55.720327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171dd82-e041-4ac5-a650-2695da2db57f admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:55.721194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171dd82-e041-4ac5-a650-2695da2db57f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:55.723492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171dd82-e041-4ac5-a650-2695da2db57f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:55.829335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa49fe-efbd-4d57-b0e0-75c0f0d637eb admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:55.845838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 834/1660] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:55 2022] POST /identity/v3/users => generated 292 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:55.851785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa49fe-efbd-4d57-b0e0-75c0f0d637eb admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:55.866874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ddb5a-5ca9-476c-b6d0-4aa9c5eb677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:55.867517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ddb5a-5ca9-476c-b6d0-4aa9c5eb677e admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:55.867800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ddb5a-5ca9-476c-b6d0-4aa9c5eb677e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:55.869874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ddb5a-5ca9-476c-b6d0-4aa9c5eb677e admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:55.873265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ddb5a-5ca9-476c-b6d0-4aa9c5eb677e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:55.878043 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ddb5a-5ca9-476c-b6d0-4aa9c5eb677e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:55.912579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171dd82-e041-4ac5-a650-2695da2db57f admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:55.925039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 827/1661] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:55.928532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171dd82-e041-4ac5-a650-2695da2db57f admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:55.945563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c9718-e987-4e48-86ee-35c4f009e207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:55.946430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9718-e987-4e48-86ee-35c4f009e207 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:55.946711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9718-e987-4e48-86ee-35c4f009e207 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:55.946979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9718-e987-4e48-86ee-35c4f009e207 admin admin] PATH_INFO: `/v3/projects/70e383c5d71e442686570ab1428d47fe/users/36d46c9c38b24d4ebb8dda86b28dbecc/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:55.985809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c9718-e987-4e48-86ee-35c4f009e207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e383c5d71e442686570ab1428d47fe, user_id=36d46c9c38b24d4ebb8dda86b28dbecc, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:55.993375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c9718-e987-4e48-86ee-35c4f009e207 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:56.043632 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ddb5a-5ca9-476c-b6d0-4aa9c5eb677e admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:56.044571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 835/1662] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:55 2022] PUT /identity/v3/projects/70e383c5d71e442686570ab1428d47fe/users/36d46c9c38b24d4ebb8dda86b28dbecc/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:56.059040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ddb5a-5ca9-476c-b6d0-4aa9c5eb677e admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:56.197533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd794ea-676d-442b-9ea5-6e03caa4f407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:56.198280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd794ea-676d-442b-9ea5-6e03caa4f407 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:56.198671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd794ea-676d-442b-9ea5-6e03caa4f407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:56.199052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd794ea-676d-442b-9ea5-6e03caa4f407 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:56.199826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd794ea-676d-442b-9ea5-6e03caa4f407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:56.201580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd794ea-676d-442b-9ea5-6e03caa4f407 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:56.233957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 828/1663] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:56.237917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c9718-e987-4e48-86ee-35c4f009e207 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:56.250929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c9718-e987-4e48-86ee-35c4f009e207 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:56.262134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2026597-bb6e-4153-b495-9b979febd2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:56.263068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2026597-bb6e-4153-b495-9b979febd2c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:56.263518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2026597-bb6e-4153-b495-9b979febd2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:56.263866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2026597-bb6e-4153-b495-9b979febd2c6 admin admin] PATH_INFO: `/v3/projects/70e383c5d71e442686570ab1428d47fe/users/36d46c9c38b24d4ebb8dda86b28dbecc/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:56.274162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2026597-bb6e-4153-b495-9b979febd2c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e383c5d71e442686570ab1428d47fe, user_id=36d46c9c38b24d4ebb8dda86b28dbecc, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:56.277740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2026597-bb6e-4153-b495-9b979febd2c6 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:56.303080 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 836/1664] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:56 2022] PUT /identity/v3/projects/70e383c5d71e442686570ab1428d47fe/users/36d46c9c38b24d4ebb8dda86b28dbecc/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:51:56.314348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9732-59ca-4129-a3db-2392fb14e2de None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:56.314890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9732-59ca-4129-a3db-2392fb14e2de None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:56.318366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9732-59ca-4129-a3db-2392fb14e2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:56.351009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2026597-bb6e-4153-b495-9b979febd2c6 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:56.353028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f2026597-bb6e-4153-b495-9b979febd2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:56.416687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-98ac9732-59ca-4129-a3db-2392fb14e2de None None] MFA Rules not processed for user `36d46c9c38b24d4ebb8dda86b28dbecc`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:56.418145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-98ac9732-59ca-4129-a3db-2392fb14e2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:56.610833 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2026597-bb6e-4153-b495-9b979febd2c6 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:56.621400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:56.623442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:56.623798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:56.624667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] PATH_INFO: `/v3/projects/a1261ce2be2f46749fe7ffb31192999a` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:56.626817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1261ce2be2f46749fe7ffb31192999a)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:56.634788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:56.652554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:56.705024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 829/1665] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:56.717025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac9732-59ca-4129-a3db-2392fb14e2de None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:56.720409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-98ac9732-59ca-4129-a3db-2392fb14e2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:56.818943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1261ce2be2f46749fe7ffb31192999a', 'request_id': 'req-e54ee840-0652-415a-996a-b53abb966ef5'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:56.841304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1261ce2be2f46749fe7ffb31192999a', 'request_id': 'req-e54ee840-0652-415a-996a-b53abb966ef5'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:56.870655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1261ce2be2f46749fe7ffb31192999a', 'request_id': 'req-e54ee840-0652-415a-996a-b53abb966ef5'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:56.886131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] The token cache is being invalidate because project a1261ce2be2f46749fe7ffb31192999a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:51:56.886732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:56.888845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e54ee840-0652-415a-996a-b53abb966ef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:51:56.891340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 837/1666] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:51:56 2022] DELETE /identity/v3/projects/a1261ce2be2f46749fe7ffb31192999a => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:51:56.895962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8508199-dd5c-4d97-8127-bfa07c6c5e2b None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:56.896379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8508199-dd5c-4d97-8127-bfa07c6c5e2b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:56.896764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8508199-dd5c-4d97-8127-bfa07c6c5e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:56.901689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:56.916913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac9732-59ca-4129-a3db-2392fb14e2de None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:56.943155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f8508199-dd5c-4d97-8127-bfa07c6c5e2b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:56.947572 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f8508199-dd5c-4d97-8127-bfa07c6c5e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:57.054462 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29a173-4076-4c39-bd2d-cbd1f1f63aa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:57.055104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29a173-4076-4c39-bd2d-cbd1f1f63aa2 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:57.055394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29a173-4076-4c39-bd2d-cbd1f1f63aa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:57.055798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29a173-4076-4c39-bd2d-cbd1f1f63aa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:57.062158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7c29a173-4076-4c39-bd2d-cbd1f1f63aa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:57.125574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 838/1667] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:57.134169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59788646-4349-4486-bfa8-5e2427204236 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:57.134684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59788646-4349-4486-bfa8-5e2427204236 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:57.135135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59788646-4349-4486-bfa8-5e2427204236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:57.178474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-59788646-4349-4486-bfa8-5e2427204236 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:57.181744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-59788646-4349-4486-bfa8-5e2427204236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:57.202551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 839/1668] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:57.202738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29a173-4076-4c39-bd2d-cbd1f1f63aa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:57.203816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29a173-4076-4c39-bd2d-cbd1f1f63aa2 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:57.205689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59788646-4349-4486-bfa8-5e2427204236 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:57.227797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59788646-4349-4486-bfa8-5e2427204236 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:57.240376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 830/1669] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:51:57.263635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff290713-f495-43fd-8e97-d9a6f298a6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:57.264368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290713-f495-43fd-8e97-d9a6f298a6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:57.271172 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290713-f495-43fd-8e97-d9a6f298a6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:57.271494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290713-f495-43fd-8e97-d9a6f298a6c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:57.272397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff290713-f495-43fd-8e97-d9a6f298a6c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:51:57.272708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff290713-f495-43fd-8e97-d9a6f298a6c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:57.286039 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff290713-f495-43fd-8e97-d9a6f298a6c7 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:57.323361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 840/1670] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:51:57.327745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29a173-4076-4c39-bd2d-cbd1f1f63aa2 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:57.359859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29a173-4076-4c39-bd2d-cbd1f1f63aa2 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:57.372285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b47c4d-98c7-45ca-937b-f96ddd8d7372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:57.372917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b47c4d-98c7-45ca-937b-f96ddd8d7372 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:57.373251 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b47c4d-98c7-45ca-937b-f96ddd8d7372 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:57.373469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b47c4d-98c7-45ca-937b-f96ddd8d7372 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:57.378123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b47c4d-98c7-45ca-937b-f96ddd8d7372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:57.380354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b47c4d-98c7-45ca-937b-f96ddd8d7372 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:57.431988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 831/1671] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:57 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:51:57.433867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff290713-f495-43fd-8e97-d9a6f298a6c7 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:57.450158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff290713-f495-43fd-8e97-d9a6f298a6c7 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:57.464586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b47c4d-98c7-45ca-937b-f96ddd8d7372 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:57.466824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-33b47c4d-98c7-45ca-937b-f96ddd8d7372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:57.586111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b47c4d-98c7-45ca-937b-f96ddd8d7372 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:57.595155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c13c0f-527c-42bf-a89c-c8a9c715a549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:57.595715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa326c7-f4ed-4b1a-9944-b485c7895e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:57.596392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c13c0f-527c-42bf-a89c-c8a9c715a549 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:57.596523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa326c7-f4ed-4b1a-9944-b485c7895e11 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:57.596735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa326c7-f4ed-4b1a-9944-b485c7895e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:57.597068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa326c7-f4ed-4b1a-9944-b485c7895e11 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:57.597933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa326c7-f4ed-4b1a-9944-b485c7895e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:57.598131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c13c0f-527c-42bf-a89c-c8a9c715a549 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:57.598562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c13c0f-527c-42bf-a89c-c8a9c715a549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:57.600101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa326c7-f4ed-4b1a-9944-b485c7895e11 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:57.603164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-88c13c0f-527c-42bf-a89c-c8a9c715a549 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:57.641638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 841/1672] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:57 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:57.643232 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa326c7-f4ed-4b1a-9944-b485c7895e11 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:57.653889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:57.663577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa326c7-f4ed-4b1a-9944-b485c7895e11 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:57.674868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2499ecc-6c9d-443b-b3ea-d2eea6643c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:57.675637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2499ecc-6c9d-443b-b3ea-d2eea6643c48 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:57.678793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2499ecc-6c9d-443b-b3ea-d2eea6643c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:57.679443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2499ecc-6c9d-443b-b3ea-d2eea6643c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:57.680457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2499ecc-6c9d-443b-b3ea-d2eea6643c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:57.682451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2499ecc-6c9d-443b-b3ea-d2eea6643c48 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:57.703600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 842/1673] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:51:57.706084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2499ecc-6c9d-443b-b3ea-d2eea6643c48 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:57.710128 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:57.728369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c13c0f-527c-42bf-a89c-c8a9c715a549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:57.730213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c13c0f-527c-42bf-a89c-c8a9c715a549 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:57.735106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2499ecc-6c9d-443b-b3ea-d2eea6643c48 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:57.745277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 832/1674] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:51:57.748577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e25f23-f66a-4c67-a5c7-c17360335e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:57.749602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e25f23-f66a-4c67-a5c7-c17360335e28 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:57.750037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e25f23-f66a-4c67-a5c7-c17360335e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:57.750460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e25f23-f66a-4c67-a5c7-c17360335e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:57.751262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e25f23-f66a-4c67-a5c7-c17360335e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:57.753101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e25f23-f66a-4c67-a5c7-c17360335e28 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:57.772126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c13c0f-527c-42bf-a89c-c8a9c715a549 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:57.774268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 843/1675] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:57.786521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c13c0f-527c-42bf-a89c-c8a9c715a549 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:57.792091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e490-edf2-4a75-9b4f-c95b7fc98007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:57.793350 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e490-edf2-4a75-9b4f-c95b7fc98007 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:57.793829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e490-edf2-4a75-9b4f-c95b7fc98007 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:57.794289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e490-edf2-4a75-9b4f-c95b7fc98007 admin admin] PATH_INFO: `/v3/projects/3f7c3d0e34d54538a20faae27cd5f8e4/users/bab817a1a0da43d0b5306fda917a227d/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:57.814138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608e490-edf2-4a75-9b4f-c95b7fc98007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f7c3d0e34d54538a20faae27cd5f8e4, user_id=bab817a1a0da43d0b5306fda917a227d, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:57.818973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608e490-edf2-4a75-9b4f-c95b7fc98007 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:57.855674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cd7d3-e5db-40fa-b36b-d23fae6fc287 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:57.855964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cd7d3-e5db-40fa-b36b-d23fae6fc287 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:57.856302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cd7d3-e5db-40fa-b36b-d23fae6fc287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:57.859173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 833/1676] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:57 2022] PUT /identity/v3/projects/3f7c3d0e34d54538a20faae27cd5f8e4/users/bab817a1a0da43d0b5306fda917a227d/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:57.896720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-578cd7d3-e5db-40fa-b36b-d23fae6fc287 None None] MFA Rules not processed for user `bab817a1a0da43d0b5306fda917a227d`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:57.898010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-578cd7d3-e5db-40fa-b36b-d23fae6fc287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:58.027943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e490-edf2-4a75-9b4f-c95b7fc98007 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:58.028475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 844/1677] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:58.157556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e490-edf2-4a75-9b4f-c95b7fc98007 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:58.168221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d74e1-f7f0-49d3-9e7b-9826338967e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:58.169049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d74e1-f7f0-49d3-9e7b-9826338967e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:58.169163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d74e1-f7f0-49d3-9e7b-9826338967e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:58.169432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d74e1-f7f0-49d3-9e7b-9826338967e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:58.170126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d74e1-f7f0-49d3-9e7b-9826338967e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:58.172470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d74e1-f7f0-49d3-9e7b-9826338967e7 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:58.260919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cd7d3-e5db-40fa-b36b-d23fae6fc287 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:58.263308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 834/1678] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:58 2022] POST /identity/v3/projects => generated 387 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:58.274719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cd7d3-e5db-40fa-b36b-d23fae6fc287 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:58.487057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d330e8a-a476-4192-a199-f11cacaa317a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:58.496513 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d330e8a-a476-4192-a199-f11cacaa317a admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:58.497329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d330e8a-a476-4192-a199-f11cacaa317a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:58.504313 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d330e8a-a476-4192-a199-f11cacaa317a admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:58.506066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d330e8a-a476-4192-a199-f11cacaa317a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:58.512771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d330e8a-a476-4192-a199-f11cacaa317a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:58.588097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 845/1679] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:58 2022] POST /identity/v3/users => generated 294 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:58.592250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d74e1-f7f0-49d3-9e7b-9826338967e7 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:58.645086 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d74e1-f7f0-49d3-9e7b-9826338967e7 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:58.666672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b679be8-f5e8-4161-b7cb-ad0879ca08f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:58.670092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b679be8-f5e8-4161-b7cb-ad0879ca08f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:58.675606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b679be8-f5e8-4161-b7cb-ad0879ca08f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:58.677985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b679be8-f5e8-4161-b7cb-ad0879ca08f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:58.684926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b679be8-f5e8-4161-b7cb-ad0879ca08f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:58.689403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b679be8-f5e8-4161-b7cb-ad0879ca08f5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:58.746941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d330e8a-a476-4192-a199-f11cacaa317a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:58.749105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 835/1680] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:51:58.762122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d330e8a-a476-4192-a199-f11cacaa317a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:58.777908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d37ec9-f6ac-41d4-8504-5d8579c13215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:58.779841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d37ec9-f6ac-41d4-8504-5d8579c13215 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:58.781306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d37ec9-f6ac-41d4-8504-5d8579c13215 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:58.781829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d37ec9-f6ac-41d4-8504-5d8579c13215 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:58.783295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d37ec9-f6ac-41d4-8504-5d8579c13215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:58.786083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d37ec9-f6ac-41d4-8504-5d8579c13215 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:58.811146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 846/1681] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:51:58.812308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b679be8-f5e8-4161-b7cb-ad0879ca08f5 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:58.824914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b679be8-f5e8-4161-b7cb-ad0879ca08f5 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:58.851830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3231632-bf8f-428b-9b3a-f4de5039670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:58.852469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3231632-bf8f-428b-9b3a-f4de5039670d admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:58.852757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3231632-bf8f-428b-9b3a-f4de5039670d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:58.853035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3231632-bf8f-428b-9b3a-f4de5039670d admin admin] PATH_INFO: `/v3/projects/c88bb75b19f947838d04c0013ee4773d/users/e2c4ac1bfaa543c6b37d796a9345eb32/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:58.894589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3231632-bf8f-428b-9b3a-f4de5039670d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88bb75b19f947838d04c0013ee4773d, user_id=e2c4ac1bfaa543c6b37d796a9345eb32, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:58.900945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3231632-bf8f-428b-9b3a-f4de5039670d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:58.945206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 836/1682] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:51:58 2022] PUT /identity/v3/projects/c88bb75b19f947838d04c0013ee4773d/users/e2c4ac1bfaa543c6b37d796a9345eb32/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:51:58.945694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909aab56-b2de-45a3-8d32-09e405da4400 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:58.946112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909aab56-b2de-45a3-8d32-09e405da4400 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:58.946519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909aab56-b2de-45a3-8d32-09e405da4400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:59.014777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-909aab56-b2de-45a3-8d32-09e405da4400 None None] MFA Rules not processed for user `e2c4ac1bfaa543c6b37d796a9345eb32`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:59.019393 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-909aab56-b2de-45a3-8d32-09e405da4400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:59.132982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:59.190922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 847/1683] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:51:59.194605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3231632-bf8f-428b-9b3a-f4de5039670d admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:59.308219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793fddb-2f88-40ba-875f-c00cdd027a21 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:59.308219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793fddb-2f88-40ba-875f-c00cdd027a21 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:59.308631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793fddb-2f88-40ba-875f-c00cdd027a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:59.444235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3231632-bf8f-428b-9b3a-f4de5039670d admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:59.451195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c793fddb-2f88-40ba-875f-c00cdd027a21 None None] MFA Rules not processed for user `89c747f18b434e469ae61755e9f639ca`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:51:59.452461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c793fddb-2f88-40ba-875f-c00cdd027a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:59.467846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a3eaa-d936-47f5-9375-1980dd3f51be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:59.468709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a3eaa-d936-47f5-9375-1980dd3f51be admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:59.473829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a3eaa-d936-47f5-9375-1980dd3f51be admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:59.474302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a3eaa-d936-47f5-9375-1980dd3f51be admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:59.475046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a3eaa-d936-47f5-9375-1980dd3f51be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:59.484538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a3eaa-d936-47f5-9375-1980dd3f51be admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:59.554566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:51:59.580278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 837/1684] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:51:59 2022] POST /identity/v3/projects => generated 385 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:51:59.584911 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a3eaa-d936-47f5-9375-1980dd3f51be admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:59.715895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 848/1685] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:51:59.744101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793fddb-2f88-40ba-875f-c00cdd027a21 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:59.748163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c793fddb-2f88-40ba-875f-c00cdd027a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:51:59.792331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a3eaa-d936-47f5-9375-1980dd3f51be admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:59.812077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26414cd7-8b65-47f4-859d-d9a1d6e5dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:59.812710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26414cd7-8b65-47f4-859d-d9a1d6e5dd31 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:59.815684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26414cd7-8b65-47f4-859d-d9a1d6e5dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:59.815684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26414cd7-8b65-47f4-859d-d9a1d6e5dd31 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:59.815684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26414cd7-8b65-47f4-859d-d9a1d6e5dd31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:59.816045 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26414cd7-8b65-47f4-859d-d9a1d6e5dd31 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:59.890558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 838/1686] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:51:59 2022] POST /identity/v3/users => generated 299 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:51:59.895895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26414cd7-8b65-47f4-859d-d9a1d6e5dd31 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:51:59.935646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26414cd7-8b65-47f4-859d-d9a1d6e5dd31 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:51:59.963700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2ae08-6fb3-4135-bcff-7786434515c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:51:59.964438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2ae08-6fb3-4135-bcff-7786434515c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:51:59.964723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2ae08-6fb3-4135-bcff-7786434515c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:51:59.965059 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2ae08-6fb3-4135-bcff-7786434515c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:51:59.965939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2ae08-6fb3-4135-bcff-7786434515c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:51:59.967773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2ae08-6fb3-4135-bcff-7786434515c8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:51:59.998965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793fddb-2f88-40ba-875f-c00cdd027a21 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:00.009913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 839/1687] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:00.016577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2ae08-6fb3-4135-bcff-7786434515c8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:00.054014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2ae08-6fb3-4135-bcff-7786434515c8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:00.079163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd54ca8-8d5e-4de1-8434-eed7b268930f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:00.079874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd54ca8-8d5e-4de1-8434-eed7b268930f admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:00.080160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd54ca8-8d5e-4de1-8434-eed7b268930f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:00.080571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd54ca8-8d5e-4de1-8434-eed7b268930f admin admin] PATH_INFO: `/v3/projects/398371e95a9e43aa81696a6a76624f66/users/ddfe900ac9214cf2994a0a95ce557688/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:00.166433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd54ca8-8d5e-4de1-8434-eed7b268930f admin admin] RBAC: Authorizing `identity:create_grant(project_id=398371e95a9e43aa81696a6a76624f66, user_id=ddfe900ac9214cf2994a0a95ce557688, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:00.169681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd54ca8-8d5e-4de1-8434-eed7b268930f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:00.221654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 840/1688] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:00 2022] PUT /identity/v3/projects/398371e95a9e43aa81696a6a76624f66/users/ddfe900ac9214cf2994a0a95ce557688/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:00.226217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd54ca8-8d5e-4de1-8434-eed7b268930f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:00.303995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50addabe-2800-41ab-b9fb-d59c17cb99a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:00.305818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50addabe-2800-41ab-b9fb-d59c17cb99a2 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:00.305950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50addabe-2800-41ab-b9fb-d59c17cb99a2 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:00.306132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50addabe-2800-41ab-b9fb-d59c17cb99a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:00.333046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:00.392924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd54ca8-8d5e-4de1-8434-eed7b268930f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:00.407747 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b86b5-eb88-4cee-b0d8-3aaa3789749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:00.409458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b86b5-eb88-4cee-b0d8-3aaa3789749f admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:00.409774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b86b5-eb88-4cee-b0d8-3aaa3789749f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:00.410045 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b86b5-eb88-4cee-b0d8-3aaa3789749f admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:00.413626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b86b5-eb88-4cee-b0d8-3aaa3789749f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:00.417527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b86b5-eb88-4cee-b0d8-3aaa3789749f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:00.441838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 841/1689] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:00.448655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b86b5-eb88-4cee-b0d8-3aaa3789749f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:00.468873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b86b5-eb88-4cee-b0d8-3aaa3789749f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:00.481179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280e3ba-97df-4e7b-9ad7-6537d0bd36bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:00.482538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280e3ba-97df-4e7b-9ad7-6537d0bd36bb admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:00.483816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280e3ba-97df-4e7b-9ad7-6537d0bd36bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:00.484109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280e3ba-97df-4e7b-9ad7-6537d0bd36bb admin admin] PATH_INFO: `/v3/projects/398371e95a9e43aa81696a6a76624f66/users/ddfe900ac9214cf2994a0a95ce557688/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:00.489903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280e3ba-97df-4e7b-9ad7-6537d0bd36bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=398371e95a9e43aa81696a6a76624f66, user_id=ddfe900ac9214cf2994a0a95ce557688, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:00.497778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280e3ba-97df-4e7b-9ad7-6537d0bd36bb admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:00.530081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 842/1690] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:00 2022] PUT /identity/v3/projects/398371e95a9e43aa81696a6a76624f66/users/ddfe900ac9214cf2994a0a95ce557688/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:00.542497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66987e6-c585-4970-85c8-d543edfb3fe7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:00.542782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66987e6-c585-4970-85c8-d543edfb3fe7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:00.543065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66987e6-c585-4970-85c8-d543edfb3fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:00.583117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a66987e6-c585-4970-85c8-d543edfb3fe7 None None] MFA Rules not processed for user `ddfe900ac9214cf2994a0a95ce557688`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:00.585311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a66987e6-c585-4970-85c8-d543edfb3fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:00.607644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50addabe-2800-41ab-b9fb-d59c17cb99a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:00.613619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50addabe-2800-41ab-b9fb-d59c17cb99a2 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:00.637635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 849/1691] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 895 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:00.649476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:00.771857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50addabe-2800-41ab-b9fb-d59c17cb99a2 service nova] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:00.774328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 843/1692] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:00.926664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50addabe-2800-41ab-b9fb-d59c17cb99a2 service nova] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:01.134123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af109eaa-0050-4928-ae95-a315253514b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:01.135000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af109eaa-0050-4928-ae95-a315253514b6 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:01.135392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af109eaa-0050-4928-ae95-a315253514b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:01.135819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af109eaa-0050-4928-ae95-a315253514b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:01.158452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af109eaa-0050-4928-ae95-a315253514b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:01.160262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af109eaa-0050-4928-ae95-a315253514b6 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:01.183822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 850/1693] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:52:01.286921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66987e6-c585-4970-85c8-d543edfb3fe7 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:01.290154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a66987e6-c585-4970-85c8-d543edfb3fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:01.572897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66987e6-c585-4970-85c8-d543edfb3fe7 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:01.590399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:01.594088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:01.595374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:01.597180 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] PATH_INFO: `/v3/users/33088e0787eb4ed5ba780f036258c082` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:01.624785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33088e0787eb4ed5ba780f036258c082)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:01.640491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:01.737585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33088e0787eb4ed5ba780f036258c082'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:01.738801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33088e0787eb4ed5ba780f036258c082'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:01.762369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33088e0787eb4ed5ba780f036258c082'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:01.792465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33088e0787eb4ed5ba780f036258c082'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:01.809194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33088e0787eb4ed5ba780f036258c082'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:01.822486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 844/1694] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:01 2022] DELETE /identity/v3/users/33088e0787eb4ed5ba780f036258c082 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:01.827194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a988d-fdb2-463f-9da6-0a7c92b750a8 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:01.827704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a988d-fdb2-463f-9da6-0a7c92b750a8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:01.827968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a988d-fdb2-463f-9da6-0a7c92b750a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:01.875045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-a58a988d-fdb2-463f-9da6-0a7c92b750a8 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:01.877953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a58a988d-fdb2-463f-9da6-0a7c92b750a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:02.092764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 851/1695] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:02.159973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:02.162513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:02.272479 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:02.305147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c063556-b5ab-4499-8dd9-b27c5934e570 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:02.318729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0236e-45ac-49e3-adf4-2e3bd7013b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:02.319375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0236e-45ac-49e3-adf4-2e3bd7013b36 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:02.319658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0236e-45ac-49e3-adf4-2e3bd7013b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:02.319928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0236e-45ac-49e3-adf4-2e3bd7013b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:02.341078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0236e-45ac-49e3-adf4-2e3bd7013b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:02.347706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0236e-45ac-49e3-adf4-2e3bd7013b36 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:02.390950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 845/1696] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:02.441120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a988d-fdb2-463f-9da6-0a7c92b750a8 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:02.442611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a58a988d-fdb2-463f-9da6-0a7c92b750a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:02.449078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:02.588123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a988d-fdb2-463f-9da6-0a7c92b750a8 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:02.598242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7745d6-f35a-4296-9b0b-827bcd6a5bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:02.601671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7745d6-f35a-4296-9b0b-827bcd6a5bfb service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:02.602191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7745d6-f35a-4296-9b0b-827bcd6a5bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:02.602629 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7745d6-f35a-4296-9b0b-827bcd6a5bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:02.605843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ad7745d6-f35a-4296-9b0b-827bcd6a5bfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:02.788534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7745d6-f35a-4296-9b0b-827bcd6a5bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:02.790412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7745d6-f35a-4296-9b0b-827bcd6a5bfb service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:02.824224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 852/1697] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:04.308504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0236e-45ac-49e3-adf4-2e3bd7013b36 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:04.310592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0aa0236e-45ac-49e3-adf4-2e3bd7013b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:04.333948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0236e-45ac-49e3-adf4-2e3bd7013b36 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:04.346873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:04.347688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:04.348164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:04.349191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] PATH_INFO: `/v3/projects/56485c17468e448cb52e2c2206e8c6c2` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:04.350700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56485c17468e448cb52e2c2206e8c6c2)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:04.352757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:04.467536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56485c17468e448cb52e2c2206e8c6c2', 'request_id': 'req-aed1613f-7734-455e-9319-908ad3b74523'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:04.492908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56485c17468e448cb52e2c2206e8c6c2', 'request_id': 'req-aed1613f-7734-455e-9319-908ad3b74523'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:04.501257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56485c17468e448cb52e2c2206e8c6c2', 'request_id': 'req-aed1613f-7734-455e-9319-908ad3b74523'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:04.513066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efa5e8-429d-461c-a998-f6370271d217 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:04.513729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efa5e8-429d-461c-a998-f6370271d217 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:04.513729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efa5e8-429d-461c-a998-f6370271d217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:04.535232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] The token cache is being invalidate because project 56485c17468e448cb52e2c2206e8c6c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:04.535692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:04.542312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-aed1613f-7734-455e-9319-908ad3b74523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:04.545681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 846/1698] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:04 2022] DELETE /identity/v3/projects/56485c17468e448cb52e2c2206e8c6c2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:04.551004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461268d-878a-411e-b3f3-0ab6d9cd0787 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:04.552061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461268d-878a-411e-b3f3-0ab6d9cd0787 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:04.552659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461268d-878a-411e-b3f3-0ab6d9cd0787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:04.582846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-75efa5e8-429d-461c-a998-f6370271d217 None None] MFA Rules not processed for user `913c969eb6274750aef880400a74e637`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:04.584124 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-75efa5e8-429d-461c-a998-f6370271d217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:04.589741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9461268d-878a-411e-b3f3-0ab6d9cd0787 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:04.591117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9461268d-878a-411e-b3f3-0ab6d9cd0787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:04.717043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 853/1699] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:04.731492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3a91f-56a2-4e6c-92b1-dcbd446ef596 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:04.731785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3a91f-56a2-4e6c-92b1-dcbd446ef596 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:04.734554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3a91f-56a2-4e6c-92b1-dcbd446ef596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:04.735212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 847/1700] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:04.743264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198706-0211-4cf1-8caa-0d4c23822d56 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:04.743702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198706-0211-4cf1-8caa-0d4c23822d56 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:04.744150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198706-0211-4cf1-8caa-0d4c23822d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:04.784194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-96198706-0211-4cf1-8caa-0d4c23822d56 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:04.786989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-96198706-0211-4cf1-8caa-0d4c23822d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:04.790468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-60d3a91f-56a2-4e6c-92b1-dcbd446ef596 None None] MFA Rules not processed for user `913c969eb6274750aef880400a74e637`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:04.796317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-60d3a91f-56a2-4e6c-92b1-dcbd446ef596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:04.809879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 848/1701] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:04.814067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198706-0211-4cf1-8caa-0d4c23822d56 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:04.825522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 854/1702] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:04.836010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198706-0211-4cf1-8caa-0d4c23822d56 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:04.838493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3a91f-56a2-4e6c-92b1-dcbd446ef596 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:04.840089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-60d3a91f-56a2-4e6c-92b1-dcbd446ef596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:04.857042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da26e0-34e5-4705-bee8-bd163bcde124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:04.857806 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da26e0-34e5-4705-bee8-bd163bcde124 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:04.858164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da26e0-34e5-4705-bee8-bd163bcde124 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:04.858503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da26e0-34e5-4705-bee8-bd163bcde124 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:04.859240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da26e0-34e5-4705-bee8-bd163bcde124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:52:04.859611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da26e0-34e5-4705-bee8-bd163bcde124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:04.861676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da26e0-34e5-4705-bee8-bd163bcde124 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:04.883941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 849/1703] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:52:04.888658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da26e0-34e5-4705-bee8-bd163bcde124 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:04.916327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da26e0-34e5-4705-bee8-bd163bcde124 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:04.928688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6b26b-adfe-4cbd-888e-59dec42f200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:04.930573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b26b-adfe-4cbd-888e-59dec42f200f admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:04.930862 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b26b-adfe-4cbd-888e-59dec42f200f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:04.931166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b26b-adfe-4cbd-888e-59dec42f200f admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:04.931839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6b26b-adfe-4cbd-888e-59dec42f200f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:04.935834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6b26b-adfe-4cbd-888e-59dec42f200f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:04.963127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:05.011989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 850/1704] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:04 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:05.015833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6b26b-adfe-4cbd-888e-59dec42f200f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:05.025473 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3a91f-56a2-4e6c-92b1-dcbd446ef596 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:05.071759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:05.174157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad6abc-fbc8-43d5-84ad-d51fb8f34788 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:05.175273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad6abc-fbc8-43d5-84ad-d51fb8f34788 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:05.176574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad6abc-fbc8-43d5-84ad-d51fb8f34788 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:05.176957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad6abc-fbc8-43d5-84ad-d51fb8f34788 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:05.188755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6b26b-adfe-4cbd-888e-59dec42f200f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:05.207465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65137187-9f5c-4141-aee0-69acbc71e844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:05.208607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65137187-9f5c-4141-aee0-69acbc71e844 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:05.208888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65137187-9f5c-4141-aee0-69acbc71e844 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:05.209161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65137187-9f5c-4141-aee0-69acbc71e844 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:05.209864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65137187-9f5c-4141-aee0-69acbc71e844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:05.211816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65137187-9f5c-4141-aee0-69acbc71e844 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:05.276427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 851/1705] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:05 2022] POST /identity/v3/users => generated 294 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:05.281346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65137187-9f5c-4141-aee0-69acbc71e844 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:05.311744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad6abc-fbc8-43d5-84ad-d51fb8f34788 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:05.313476 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad6abc-fbc8-43d5-84ad-d51fb8f34788 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:05.314456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65137187-9f5c-4141-aee0-69acbc71e844 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:05.337105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 855/1706] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:05.338502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12953dd-d527-46a5-9a02-5dd77c116ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:05.343981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12953dd-d527-46a5-9a02-5dd77c116ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:05.346469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12953dd-d527-46a5-9a02-5dd77c116ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:05.346912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12953dd-d527-46a5-9a02-5dd77c116ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:05.349923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12953dd-d527-46a5-9a02-5dd77c116ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:05.352471 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12953dd-d527-46a5-9a02-5dd77c116ab8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:05.378655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 852/1707] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:05.381383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad6abc-fbc8-43d5-84ad-d51fb8f34788 service nova] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:05.393644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad6abc-fbc8-43d5-84ad-d51fb8f34788 service nova] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:05.400236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102115d5-cf6a-4b3e-996e-3ad8a026740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:05.401168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102115d5-cf6a-4b3e-996e-3ad8a026740e admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:05.401583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102115d5-cf6a-4b3e-996e-3ad8a026740e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:05.401959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102115d5-cf6a-4b3e-996e-3ad8a026740e admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:05.402818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102115d5-cf6a-4b3e-996e-3ad8a026740e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:05.404710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102115d5-cf6a-4b3e-996e-3ad8a026740e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:05.425122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1312e7-14e4-4d40-af4f-a6e545f158dd None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:05.426277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1312e7-14e4-4d40-af4f-a6e545f158dd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:05.427756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1312e7-14e4-4d40-af4f-a6e545f158dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:05.451151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 856/1708] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:05.455569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102115d5-cf6a-4b3e-996e-3ad8a026740e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:05.470731 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2a1312e7-14e4-4d40-af4f-a6e545f158dd None None] MFA Rules not processed for user `89c747f18b434e469ae61755e9f639ca`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:05.475669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2a1312e7-14e4-4d40-af4f-a6e545f158dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:05.478633 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102115d5-cf6a-4b3e-996e-3ad8a026740e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:05.487326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa84bc2-cd3d-4c92-862e-019bc0a38892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:05.488078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa84bc2-cd3d-4c92-862e-019bc0a38892 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:05.488405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa84bc2-cd3d-4c92-862e-019bc0a38892 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:05.488687 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa84bc2-cd3d-4c92-862e-019bc0a38892 admin admin] PATH_INFO: `/v3/projects/b1572107b31e46d4acbeae8cfee7f4a3/users/37fab8afeb8642f7af84f504721d2276/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:05.531287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa84bc2-cd3d-4c92-862e-019bc0a38892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1572107b31e46d4acbeae8cfee7f4a3, user_id=37fab8afeb8642f7af84f504721d2276, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:05.541974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa84bc2-cd3d-4c92-862e-019bc0a38892 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:05.622355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 853/1710] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:05.622355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564852ac-86ce-4420-a4bd-e2cda2484e06 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:05.622355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564852ac-86ce-4420-a4bd-e2cda2484e06 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:05.622355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564852ac-86ce-4420-a4bd-e2cda2484e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:05.625668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 857/1710] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:05 2022] PUT /identity/v3/projects/b1572107b31e46d4acbeae8cfee7f4a3/users/37fab8afeb8642f7af84f504721d2276/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:05.627898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e91cf-6410-4044-a591-1abdebcd15a6 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:05.630448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e91cf-6410-4044-a591-1abdebcd15a6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:05.632361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e91cf-6410-4044-a591-1abdebcd15a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:05.688942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-564852ac-86ce-4420-a4bd-e2cda2484e06 None None] MFA Rules not processed for user `335dd8706d4c402e98605479b57d059a`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:05.690322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-564852ac-86ce-4420-a4bd-e2cda2484e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:05.690688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-181e91cf-6410-4044-a591-1abdebcd15a6 None None] MFA Rules not processed for user `37fab8afeb8642f7af84f504721d2276`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:05.699329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-181e91cf-6410-4044-a591-1abdebcd15a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:05.767005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:05.825071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 854/1711] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:05.829343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 858/1712] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:05.836362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f799c-750a-4caf-9caa-77a279a9ea1c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:05.836908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f799c-750a-4caf-9caa-77a279a9ea1c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:05.837367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f799c-750a-4caf-9caa-77a279a9ea1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:05.840569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e91cf-6410-4044-a591-1abdebcd15a6 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:05.884407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6f4f799c-750a-4caf-9caa-77a279a9ea1c None None] MFA Rules not processed for user `335dd8706d4c402e98605479b57d059a`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:05.888766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6f4f799c-750a-4caf-9caa-77a279a9ea1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:05.913629 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 855/1713] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:06.001444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e91cf-6410-4044-a591-1abdebcd15a6 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:06.015108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee431719-e1de-4e25-ac88-9f4843d23f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:06.017028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee431719-e1de-4e25-ac88-9f4843d23f05 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:06.017312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee431719-e1de-4e25-ac88-9f4843d23f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:06.017614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee431719-e1de-4e25-ac88-9f4843d23f05 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:06.018284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee431719-e1de-4e25-ac88-9f4843d23f05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:06.020678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee431719-e1de-4e25-ac88-9f4843d23f05 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:06.116455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 859/1714] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:05 2022] POST /identity/v3/projects => generated 387 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:06.120802 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f799c-750a-4caf-9caa-77a279a9ea1c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:06.146292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f799c-750a-4caf-9caa-77a279a9ea1c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:06.222670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:06.300780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429de11a-8060-47ac-9bf1-d292244b501f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:06.301906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429de11a-8060-47ac-9bf1-d292244b501f admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:06.302394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429de11a-8060-47ac-9bf1-d292244b501f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:06.303166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429de11a-8060-47ac-9bf1-d292244b501f admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:06.305653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429de11a-8060-47ac-9bf1-d292244b501f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:06.311064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429de11a-8060-47ac-9bf1-d292244b501f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:06.368885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 856/1715] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:06 2022] POST /identity/v3/users => generated 300 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:06.377440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee431719-e1de-4e25-ac88-9f4843d23f05 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:06.399954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05267ef-4d64-4da1-8556-770f3376c1c2 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:06.400827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05267ef-4d64-4da1-8556-770f3376c1c2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:06.401265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05267ef-4d64-4da1-8556-770f3376c1c2 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:06.403572 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 857/1716] 10.222.0.46 () {54 vars in 912 bytes} [Mon Oct 3 14:52:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 14:52:06.406067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee431719-e1de-4e25-ac88-9f4843d23f05 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:06.421760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f53d0-96f9-4657-8a06-181af540b69b None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:06.422965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f53d0-96f9-4657-8a06-181af540b69b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:06.423821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f53d0-96f9-4657-8a06-181af540b69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:06.438575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7712364-f773-4b8b-8496-0b882bc1f0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:06.440372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7712364-f773-4b8b-8496-0b882bc1f0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:06.440664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7712364-f773-4b8b-8496-0b882bc1f0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:06.440994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7712364-f773-4b8b-8496-0b882bc1f0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:06.442760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7712364-f773-4b8b-8496-0b882bc1f0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:06.449532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7712364-f773-4b8b-8496-0b882bc1f0d1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:06.484773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 860/1717] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:06.485719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7712364-f773-4b8b-8496-0b882bc1f0d1 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:06.499044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7712364-f773-4b8b-8496-0b882bc1f0d1 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:06.518001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fea01-e7c4-413d-84ee-873f89b88d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:06.518779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fea01-e7c4-413d-84ee-873f89b88d24 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:06.519487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fea01-e7c4-413d-84ee-873f89b88d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:06.519487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fea01-e7c4-413d-84ee-873f89b88d24 admin admin] PATH_INFO: `/v3/projects/6e213de3674d43fd929731e751e4ee0e/users/eb8f376fe8b8490b992d6ae3d3c18d0f/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:06.540577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-555f53d0-96f9-4657-8a06-181af540b69b None None] MFA Rules not processed for user `05e9009410a54ce3ab88fa0cac87a7f9`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:06.540577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-555f53d0-96f9-4657-8a06-181af540b69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:06.546655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fea01-e7c4-413d-84ee-873f89b88d24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e213de3674d43fd929731e751e4ee0e, user_id=eb8f376fe8b8490b992d6ae3d3c18d0f, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:06.557978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fea01-e7c4-413d-84ee-873f89b88d24 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:06.610542 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 861/1718] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:06 2022] PUT /identity/v3/projects/6e213de3674d43fd929731e751e4ee0e/users/eb8f376fe8b8490b992d6ae3d3c18d0f/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:06.611679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fea01-e7c4-413d-84ee-873f89b88d24 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:06.677887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 858/1719] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 14:52:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:06.704456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f53d0-96f9-4657-8a06-181af540b69b None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:06.713915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-555f53d0-96f9-4657-8a06-181af540b69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:06.728422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f53d0-96f9-4657-8a06-181af540b69b None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:06.778515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fea01-e7c4-413d-84ee-873f89b88d24 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:06.794566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a64a8-cd07-413d-85f8-cdbe9de4dea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:06.795204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a64a8-cd07-413d-85f8-cdbe9de4dea1 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:06.795483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a64a8-cd07-413d-85f8-cdbe9de4dea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:06.795752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a64a8-cd07-413d-85f8-cdbe9de4dea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:06.796412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a64a8-cd07-413d-85f8-cdbe9de4dea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:06.800564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a64a8-cd07-413d-85f8-cdbe9de4dea1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:06.826926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 862/1720] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:06.828267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c380d62-bd2e-424d-bc53-643d062bcaf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:06.832230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c380d62-bd2e-424d-bc53-643d062bcaf5 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:06.833281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c380d62-bd2e-424d-bc53-643d062bcaf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:06.833698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a64a8-cd07-413d-85f8-cdbe9de4dea1 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:06.834054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c380d62-bd2e-424d-bc53-643d062bcaf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:06.855154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c380d62-bd2e-424d-bc53-643d062bcaf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:06.857028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c380d62-bd2e-424d-bc53-643d062bcaf5 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:06.859839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a64a8-cd07-413d-85f8-cdbe9de4dea1 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:06.873197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b071b-d044-46de-8f76-eddbe23f3c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:06.874885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b071b-d044-46de-8f76-eddbe23f3c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:06.875342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b071b-d044-46de-8f76-eddbe23f3c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:06.875943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 859/1721] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:06.887870 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b071b-d044-46de-8f76-eddbe23f3c3c admin admin] PATH_INFO: `/v3/projects/6e213de3674d43fd929731e751e4ee0e/users/eb8f376fe8b8490b992d6ae3d3c18d0f/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:06.891159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b071b-d044-46de-8f76-eddbe23f3c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e213de3674d43fd929731e751e4ee0e, user_id=eb8f376fe8b8490b992d6ae3d3c18d0f, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:06.906820 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b071b-d044-46de-8f76-eddbe23f3c3c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:06.931424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 863/1722] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:06 2022] PUT /identity/v3/projects/6e213de3674d43fd929731e751e4ee0e/users/eb8f376fe8b8490b992d6ae3d3c18d0f/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:06.942758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5486-3438-4092-ae9c-e21a491fb1a2 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:06.943053 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5486-3438-4092-ae9c-e21a491fb1a2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:06.943331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5486-3438-4092-ae9c-e21a491fb1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:07.001063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-8c1b5486-3438-4092-ae9c-e21a491fb1a2 None None] MFA Rules not processed for user `eb8f376fe8b8490b992d6ae3d3c18d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:07.002343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8c1b5486-3438-4092-ae9c-e21a491fb1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:07.120690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:07.165865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 860/1723] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:07.176835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b071b-d044-46de-8f76-eddbe23f3c3c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:07.180209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6d0b071b-d044-46de-8f76-eddbe23f3c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:07.195155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:07.342408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b071b-d044-46de-8f76-eddbe23f3c3c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:07.352216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a94ed0-6297-4118-9179-48b6089e7ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:07.352845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a94ed0-6297-4118-9179-48b6089e7ddc service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:07.353119 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a94ed0-6297-4118-9179-48b6089e7ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:07.353385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a94ed0-6297-4118-9179-48b6089e7ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:07.373567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a94ed0-6297-4118-9179-48b6089e7ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:07.375002 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a94ed0-6297-4118-9179-48b6089e7ddc service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:07.428581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 864/1724] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:07.539317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b5486-3438-4092-ae9c-e21a491fb1a2 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:07.633795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a94ed0-6297-4118-9179-48b6089e7ddc service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:07.772537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b5486-3438-4092-ae9c-e21a491fb1a2 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:07.794098 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbad7bb-a308-4110-b5ed-5cc53088d25f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:07.798500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad7bb-a308-4110-b5ed-5cc53088d25f service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:07.802281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad7bb-a308-4110-b5ed-5cc53088d25f service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:07.802637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad7bb-a308-4110-b5ed-5cc53088d25f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:07.828987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a94ed0-6297-4118-9179-48b6089e7ddc service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:07.840540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:07.841346 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:07.841798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:07.842164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] PATH_INFO: `/v3/users/37fab8afeb8642f7af84f504721d2276` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:07.844841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=37fab8afeb8642f7af84f504721d2276)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:07.848802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:07.942822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37fab8afeb8642f7af84f504721d2276'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:07.943925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37fab8afeb8642f7af84f504721d2276'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:07.955582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbad7bb-a308-4110-b5ed-5cc53088d25f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:07.959928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbad7bb-a308-4110-b5ed-5cc53088d25f service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:07.962493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37fab8afeb8642f7af84f504721d2276'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:07.978899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37fab8afeb8642f7af84f504721d2276'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:07.979650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37fab8afeb8642f7af84f504721d2276'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:07.997819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 861/1725] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:07.999308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1276f-994c-4148-96ef-2696062f7bfb None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:08.003095 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1276f-994c-4148-96ef-2696062f7bfb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:08.003787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1276f-994c-4148-96ef-2696062f7bfb None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:08.007077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 865/1726] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:07 2022] DELETE /identity/v3/users/37fab8afeb8642f7af84f504721d2276 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:08.008216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:08.010633 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:08.010877 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 862/1727] 10.222.0.46 () {54 vars in 912 bytes} [Mon Oct 3 14:52:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:52:08.015743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260c3e-5fad-4aaf-bd81-5bdad178a50b None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:08.016256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260c3e-5fad-4aaf-bd81-5bdad178a50b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:08.016544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260c3e-5fad-4aaf-bd81-5bdad178a50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:08.062079 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:08.063848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6b260c3e-5fad-4aaf-bd81-5bdad178a50b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:08.066574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6b260c3e-5fad-4aaf-bd81-5bdad178a50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:08.182336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c7f22-643a-49a5-8908-ac8bcbb93d3b admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:08.197832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36c1de-79b6-4bc6-8ea6-3e0ba5c09f6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:08.198876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36c1de-79b6-4bc6-8ea6-3e0ba5c09f6b service placement] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:08.200043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36c1de-79b6-4bc6-8ea6-3e0ba5c09f6b service placement] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:08.200544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36c1de-79b6-4bc6-8ea6-3e0ba5c09f6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:08.207263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6c36c1de-79b6-4bc6-8ea6-3e0ba5c09f6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:08.240391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 863/1728] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:08.240391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641869c-fa29-4825-826c-c92d82e43af8 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:08.240391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641869c-fa29-4825-826c-c92d82e43af8 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:08.242283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641869c-fa29-4825-826c-c92d82e43af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:08.253714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36c1de-79b6-4bc6-8ea6-3e0ba5c09f6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:08.256170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36c1de-79b6-4bc6-8ea6-3e0ba5c09f6b service placement] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:08.274214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 866/1729] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:08.294442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36c1de-79b6-4bc6-8ea6-3e0ba5c09f6b service placement] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:08.296087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6c36c1de-79b6-4bc6-8ea6-3e0ba5c09f6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:08.300396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2641869c-fa29-4825-826c-c92d82e43af8 None None] MFA Rules not processed for user `05e9009410a54ce3ab88fa0cac87a7f9`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:08.301723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2641869c-fa29-4825-826c-c92d82e43af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:08.358603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:08.459607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36c1de-79b6-4bc6-8ea6-3e0ba5c09f6b service placement] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:08.469963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18507255-07f1-4a3b-bbe4-acf98c2f7dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:08.472146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18507255-07f1-4a3b-bbe4-acf98c2f7dda service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:08.474050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18507255-07f1-4a3b-bbe4-acf98c2f7dda service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:08.474173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 864/1730] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:08.474616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18507255-07f1-4a3b-bbe4-acf98c2f7dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:08.512918 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18507255-07f1-4a3b-bbe4-acf98c2f7dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:08.518940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18507255-07f1-4a3b-bbe4-acf98c2f7dda service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:08.561186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 867/1731] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:52:08.576627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641869c-fa29-4825-826c-c92d82e43af8 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:08.587943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641869c-fa29-4825-826c-c92d82e43af8 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:08.598898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fd38e-976d-4aff-ab32-b0bee024e0c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:08.599782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fd38e-976d-4aff-ab32-b0bee024e0c5 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:08.600197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fd38e-976d-4aff-ab32-b0bee024e0c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:08.600782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fd38e-976d-4aff-ab32-b0bee024e0c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:08.637124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fd38e-976d-4aff-ab32-b0bee024e0c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:08.638870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fd38e-976d-4aff-ab32-b0bee024e0c5 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:08.655397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 865/1732] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:09.747046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42b4b9-631d-4313-8eb7-b1f8bf157898 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:09.747604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42b4b9-631d-4313-8eb7-b1f8bf157898 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:09.747604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42b4b9-631d-4313-8eb7-b1f8bf157898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:09.780483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-cf42b4b9-631d-4313-8eb7-b1f8bf157898 None None] MFA Rules not processed for user `335dd8706d4c402e98605479b57d059a`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:09.783189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-cf42b4b9-631d-4313-8eb7-b1f8bf157898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:09.948120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 868/1733] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:10.475033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64895c-375f-484f-be80-a3b39a204277 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:10.475733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64895c-375f-484f-be80-a3b39a204277 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:10.475733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64895c-375f-484f-be80-a3b39a204277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:10.533561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc58a4-4dec-469e-8630-d71ade4c2ba7 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:10.533856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc58a4-4dec-469e-8630-d71ade4c2ba7 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:10.534135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc58a4-4dec-469e-8630-d71ade4c2ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:10.555847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6a64895c-375f-484f-be80-a3b39a204277 None None] MFA Rules not processed for user `9e5325f010ae4359bd6b514a1995614d`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:10.557202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6a64895c-375f-484f-be80-a3b39a204277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:10.662706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-3fbc58a4-4dec-469e-8630-d71ade4c2ba7 None None] MFA Rules not processed for user `0be78e1abb3947a2ac5881b9fd2f6fae`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:10.670632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3fbc58a4-4dec-469e-8630-d71ade4c2ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:10.765805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 866/1734] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:10.878014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:10.930667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64895c-375f-484f-be80-a3b39a204277 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:10.932438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6a64895c-375f-484f-be80-a3b39a204277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:10.936974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 869/1735] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:11.173778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64895c-375f-484f-be80-a3b39a204277 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:11.232940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239cdf0-d0ff-4ca6-a66a-2615e3d9f7e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:11.238698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239cdf0-d0ff-4ca6-a66a-2615e3d9f7e1 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:11.238966 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239cdf0-d0ff-4ca6-a66a-2615e3d9f7e1 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:11.239239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239cdf0-d0ff-4ca6-a66a-2615e3d9f7e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:11.285921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239cdf0-d0ff-4ca6-a66a-2615e3d9f7e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:11.292896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239cdf0-d0ff-4ca6-a66a-2615e3d9f7e1 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:11.334726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc58a4-4dec-469e-8630-d71ade4c2ba7 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:11.337012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3fbc58a4-4dec-469e-8630-d71ade4c2ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:11.349892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 867/1736] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:52:11.392273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc58a4-4dec-469e-8630-d71ade4c2ba7 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:11.441616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:11.442756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:11.443039 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:11.443310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] PATH_INFO: `/v3/projects/b1572107b31e46d4acbeae8cfee7f4a3` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:11.454700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1572107b31e46d4acbeae8cfee7f4a3)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:11.456448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:11.595924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239cdf0-d0ff-4ca6-a66a-2615e3d9f7e1 service nova] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:11.614099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239cdf0-d0ff-4ca6-a66a-2615e3d9f7e1 service nova] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:11.653058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1572107b31e46d4acbeae8cfee7f4a3', 'request_id': 'req-bf19db24-9124-413f-9546-009e14b5397e'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:11.677272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1572107b31e46d4acbeae8cfee7f4a3', 'request_id': 'req-bf19db24-9124-413f-9546-009e14b5397e'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:11.689808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1572107b31e46d4acbeae8cfee7f4a3', 'request_id': 'req-bf19db24-9124-413f-9546-009e14b5397e'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:11.710430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] The token cache is being invalidate because project b1572107b31e46d4acbeae8cfee7f4a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:11.712633 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:11.714899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:11.723949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 870/1737] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:11 2022] DELETE /identity/v3/projects/b1572107b31e46d4acbeae8cfee7f4a3 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:11.727234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:11.732020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:11.783237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:11.966281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c35127-e4d5-44e9-ab23-8b5606058512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:11.967485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c35127-e4d5-44e9-ab23-8b5606058512 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:11.967981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c35127-e4d5-44e9-ab23-8b5606058512 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:11.968452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c35127-e4d5-44e9-ab23-8b5606058512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:11.973915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-74c35127-e4d5-44e9-ab23-8b5606058512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:12.006563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19db24-9124-413f-9546-009e14b5397e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:12.037332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:12.038094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:12.038407 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:12.038647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] PATH_INFO: `/v3/users/eb8f376fe8b8490b992d6ae3d3c18d0f` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:12.040191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb8f376fe8b8490b992d6ae3d3c18d0f)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:12.042259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:12.181091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c35127-e4d5-44e9-ab23-8b5606058512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:12.183609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c35127-e4d5-44e9-ab23-8b5606058512 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:12.237580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 868/1738] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:12.244004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb8f376fe8b8490b992d6ae3d3c18d0f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:12.245149 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb8f376fe8b8490b992d6ae3d3c18d0f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:12.289421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb8f376fe8b8490b992d6ae3d3c18d0f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:12.316181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb8f376fe8b8490b992d6ae3d3c18d0f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:12.317628 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b25be431-8adb-416f-96b6-793288663a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb8f376fe8b8490b992d6ae3d3c18d0f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:12.370229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:12.414850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 871/1739] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:11 2022] DELETE /identity/v3/users/eb8f376fe8b8490b992d6ae3d3c18d0f => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:13.457056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c35127-e4d5-44e9-ab23-8b5606058512 service nova] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:13.479346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c35127-e4d5-44e9-ab23-8b5606058512 service nova] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:13.619037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:13.619710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:13.622809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:13.623193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] PATH_INFO: `/v3/projects/6e213de3674d43fd929731e751e4ee0e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:13.627023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e213de3674d43fd929731e751e4ee0e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:13.634101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:13.933709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e213de3674d43fd929731e751e4ee0e', 'request_id': 'req-7321a68a-2647-4189-9cab-1a142c9baa82'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:13.942554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258cc75-a0a1-4ee7-bf6a-f07402030090 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:13.943096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258cc75-a0a1-4ee7-bf6a-f07402030090 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:13.943096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258cc75-a0a1-4ee7-bf6a-f07402030090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:13.962406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e213de3674d43fd929731e751e4ee0e', 'request_id': 'req-7321a68a-2647-4189-9cab-1a142c9baa82'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:14.010933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e213de3674d43fd929731e751e4ee0e', 'request_id': 'req-7321a68a-2647-4189-9cab-1a142c9baa82'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:14.074901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:14.098633 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e258cc75-a0a1-4ee7-bf6a-f07402030090 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:14.099205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] The token cache is being invalidate because project 6e213de3674d43fd929731e751e4ee0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:14.103239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e258cc75-a0a1-4ee7-bf6a-f07402030090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:14.105394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:14.107957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7321a68a-2647-4189-9cab-1a142c9baa82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:14.117763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 869/1740] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:13 2022] DELETE /identity/v3/projects/6e213de3674d43fd929731e751e4ee0e => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:14.123878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c1bab-ea59-45f7-9bd8-c19490890abe None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:14.124323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c1bab-ea59-45f7-9bd8-c19490890abe None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:14.124457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c1bab-ea59-45f7-9bd8-c19490890abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:14.224237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-806c1bab-ea59-45f7-9bd8-c19490890abe None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:14.225893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-806c1bab-ea59-45f7-9bd8-c19490890abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:14.450618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 870/1741] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:14.462596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48345bf9-c586-4e0d-a656-3e4a8ab249ea None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:14.464964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48345bf9-c586-4e0d-a656-3e4a8ab249ea None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:14.465263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48345bf9-c586-4e0d-a656-3e4a8ab249ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:14.472302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 872/1742] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 531 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:14.559600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-48345bf9-c586-4e0d-a656-3e4a8ab249ea None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:14.568449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-48345bf9-c586-4e0d-a656-3e4a8ab249ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:14.608153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 871/1743] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:14.612808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258cc75-a0a1-4ee7-bf6a-f07402030090 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:14.636856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258cc75-a0a1-4ee7-bf6a-f07402030090 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:14.647367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675169b0-74db-4ddb-9eeb-e1369a8a7423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:14.648399 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675169b0-74db-4ddb-9eeb-e1369a8a7423 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:14.648399 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675169b0-74db-4ddb-9eeb-e1369a8a7423 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:14.648602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675169b0-74db-4ddb-9eeb-e1369a8a7423 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:14.650344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675169b0-74db-4ddb-9eeb-e1369a8a7423 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:52:14.650691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675169b0-74db-4ddb-9eeb-e1369a8a7423 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:14.652377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675169b0-74db-4ddb-9eeb-e1369a8a7423 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:14.697844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 873/1744] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:52:14.702266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48345bf9-c586-4e0d-a656-3e4a8ab249ea None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:14.712099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48345bf9-c586-4e0d-a656-3e4a8ab249ea None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:14.723558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cea6ae-7a5f-4f1e-868b-dc649d264c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:14.724461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cea6ae-7a5f-4f1e-868b-dc649d264c06 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:14.724854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cea6ae-7a5f-4f1e-868b-dc649d264c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:14.725380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cea6ae-7a5f-4f1e-868b-dc649d264c06 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:14.727241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cea6ae-7a5f-4f1e-868b-dc649d264c06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:14.728816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cea6ae-7a5f-4f1e-868b-dc649d264c06 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:14.743596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675169b0-74db-4ddb-9eeb-e1369a8a7423 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:14.745784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-675169b0-74db-4ddb-9eeb-e1369a8a7423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:14.776113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 872/1745] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:14 2022] POST /identity/v3/projects => generated 401 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:14.777255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cea6ae-7a5f-4f1e-868b-dc649d264c06 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:14.990046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cea6ae-7a5f-4f1e-868b-dc649d264c06 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:15.019075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfea4d8-c863-46e6-9327-4b817e758e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:15.023040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfea4d8-c863-46e6-9327-4b817e758e7c admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:15.023412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675169b0-74db-4ddb-9eeb-e1369a8a7423 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:15.023933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfea4d8-c863-46e6-9327-4b817e758e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:15.023933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfea4d8-c863-46e6-9327-4b817e758e7c admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:15.028831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfea4d8-c863-46e6-9327-4b817e758e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:15.035063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfea4d8-c863-46e6-9327-4b817e758e7c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:15.108382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:15.114440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 873/1746] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:14 2022] POST /identity/v3/users => generated 301 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:15.115633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfea4d8-c863-46e6-9327-4b817e758e7c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:15.117485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7cfea4d8-c863-46e6-9327-4b817e758e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:15.195219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ef2f8-ac5c-41f1-95cb-c66de3648c5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:15.195860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ef2f8-ac5c-41f1-95cb-c66de3648c5b service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:15.196115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ef2f8-ac5c-41f1-95cb-c66de3648c5b service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:15.196384 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ef2f8-ac5c-41f1-95cb-c66de3648c5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:15.342516 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfea4d8-c863-46e6-9327-4b817e758e7c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:15.355266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53960090-0533-4594-9ed7-2c4f8d8de935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:15.357059 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53960090-0533-4594-9ed7-2c4f8d8de935 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:15.357353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53960090-0533-4594-9ed7-2c4f8d8de935 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:15.357666 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53960090-0533-4594-9ed7-2c4f8d8de935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:15.364970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-53960090-0533-4594-9ed7-2c4f8d8de935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:15.396799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:15.476924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ef2f8-ac5c-41f1-95cb-c66de3648c5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:15.478791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ef2f8-ac5c-41f1-95cb-c66de3648c5b service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:15.509042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 874/1747] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 14:52:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 765 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 14:52:15.551766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ef2f8-ac5c-41f1-95cb-c66de3648c5b service swift] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:15.558361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53960090-0533-4594-9ed7-2c4f8d8de935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:15.592397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53960090-0533-4594-9ed7-2c4f8d8de935 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:15.594361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ef2f8-ac5c-41f1-95cb-c66de3648c5b service swift] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:15.621914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64a217-8c6a-4ff7-bf5a-f383fd04fccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:15.622552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64a217-8c6a-4ff7-bf5a-f383fd04fccb admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:15.622830 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64a217-8c6a-4ff7-bf5a-f383fd04fccb admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:15.623096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64a217-8c6a-4ff7-bf5a-f383fd04fccb admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:15.624193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64a217-8c6a-4ff7-bf5a-f383fd04fccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:15.625849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64a217-8c6a-4ff7-bf5a-f383fd04fccb admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:15.665799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 874/1748] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:15.681284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 875/1749] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:15.682647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53960090-0533-4594-9ed7-2c4f8d8de935 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:15.764872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53960090-0533-4594-9ed7-2c4f8d8de935 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:15.797684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1b460-dc45-46ed-8bcf-c3e8a3ff7024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:15.801335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b460-dc45-46ed-8bcf-c3e8a3ff7024 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:15.804871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b460-dc45-46ed-8bcf-c3e8a3ff7024 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:15.807962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b460-dc45-46ed-8bcf-c3e8a3ff7024 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:15.807962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1b460-dc45-46ed-8bcf-c3e8a3ff7024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:15.812843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1b460-dc45-46ed-8bcf-c3e8a3ff7024 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:15.869617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64a217-8c6a-4ff7-bf5a-f383fd04fccb admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:15.874691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 875/1750] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:15.893329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64a217-8c6a-4ff7-bf5a-f383fd04fccb admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:15.949188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5cc1-1616-4d7a-8969-2c1e47f6de09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:15.951623 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5cc1-1616-4d7a-8969-2c1e47f6de09 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:15.952019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5cc1-1616-4d7a-8969-2c1e47f6de09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:15.952304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5cc1-1616-4d7a-8969-2c1e47f6de09 admin admin] PATH_INFO: `/v3/projects/c014246e603e463fbc02d7261ab9db4f/users/3d230304235543449d4ca3bd297f649e/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:16.012262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef5cc1-1616-4d7a-8969-2c1e47f6de09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c014246e603e463fbc02d7261ab9db4f, user_id=3d230304235543449d4ca3bd297f649e, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:16.025144 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef5cc1-1616-4d7a-8969-2c1e47f6de09 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:16.078519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 876/1751] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:15 2022] PUT /identity/v3/projects/c014246e603e463fbc02d7261ab9db4f/users/3d230304235543449d4ca3bd297f649e/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:16.097057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94824bc5-1985-402e-b7c3-cf1e0ffe6382 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:16.098711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94824bc5-1985-402e-b7c3-cf1e0ffe6382 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:16.098711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94824bc5-1985-402e-b7c3-cf1e0ffe6382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:16.232765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-94824bc5-1985-402e-b7c3-cf1e0ffe6382 None None] MFA Rules not processed for user `3d230304235543449d4ca3bd297f649e`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:16.243676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-94824bc5-1985-402e-b7c3-cf1e0ffe6382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:16.604858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 876/1752] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:16 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:16.605614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5cc1-1616-4d7a-8969-2c1e47f6de09 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:16.978094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5cc1-1616-4d7a-8969-2c1e47f6de09 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:17.010722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9129f3-b163-44e9-b5ae-099e1b10b4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:17.015729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9129f3-b163-44e9-b5ae-099e1b10b4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:17.016579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9129f3-b163-44e9-b5ae-099e1b10b4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:17.018076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9129f3-b163-44e9-b5ae-099e1b10b4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:17.029836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9129f3-b163-44e9-b5ae-099e1b10b4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:17.037983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9129f3-b163-44e9-b5ae-099e1b10b4a9 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:17.165279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 877/1753] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:16 2022] POST /identity/v3/projects => generated 401 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:17.170196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94824bc5-1985-402e-b7c3-cf1e0ffe6382 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:17.268068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94824bc5-1985-402e-b7c3-cf1e0ffe6382 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:17.527140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68cd3-1a89-4858-929f-d17c3a5eca6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:17.531975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68cd3-1a89-4858-929f-d17c3a5eca6c admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:17.534357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68cd3-1a89-4858-929f-d17c3a5eca6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:17.534673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68cd3-1a89-4858-929f-d17c3a5eca6c admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:17.538613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b68cd3-1a89-4858-929f-d17c3a5eca6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:17.545622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b68cd3-1a89-4858-929f-d17c3a5eca6c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:17.651425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 877/1754] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:17 2022] POST /identity/v3/users => generated 307 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:17.655886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9129f3-b163-44e9-b5ae-099e1b10b4a9 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:17.680708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9129f3-b163-44e9-b5ae-099e1b10b4a9 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:17.701293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca496a0-e84c-49e6-bd28-a5c9c293b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:17.702353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca496a0-e84c-49e6-bd28-a5c9c293b3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:17.702839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca496a0-e84c-49e6-bd28-a5c9c293b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:17.703322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca496a0-e84c-49e6-bd28-a5c9c293b3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:17.704203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca496a0-e84c-49e6-bd28-a5c9c293b3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:17.706083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca496a0-e84c-49e6-bd28-a5c9c293b3f8 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:17.747489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 878/1755] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:17.755056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68cd3-1a89-4858-929f-d17c3a5eca6c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:17.768953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68cd3-1a89-4858-929f-d17c3a5eca6c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:17.814732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bbaa1-e52d-4899-a540-0cf0183ed2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:17.816174 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bbaa1-e52d-4899-a540-0cf0183ed2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:17.817262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bbaa1-e52d-4899-a540-0cf0183ed2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:17.817892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bbaa1-e52d-4899-a540-0cf0183ed2fd admin admin] PATH_INFO: `/v3/projects/4b54ce21c38c458bb90ef6ec41dd89f8/users/a0417a79561643a9b336d3811b1fe0fc/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:17.880568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bbaa1-e52d-4899-a540-0cf0183ed2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b54ce21c38c458bb90ef6ec41dd89f8, user_id=a0417a79561643a9b336d3811b1fe0fc, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:17.886626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bbaa1-e52d-4899-a540-0cf0183ed2fd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:17.911569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:17.920619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 878/1756] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:17 2022] PUT /identity/v3/projects/4b54ce21c38c458bb90ef6ec41dd89f8/users/a0417a79561643a9b336d3811b1fe0fc/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:17.921641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca496a0-e84c-49e6-bd28-a5c9c293b3f8 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:17.955244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca496a0-e84c-49e6-bd28-a5c9c293b3f8 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:17.998468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:18.162876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7942bfe-b6aa-4d04-9634-d80aeb333408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:18.165303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7942bfe-b6aa-4d04-9634-d80aeb333408 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:18.166253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7942bfe-b6aa-4d04-9634-d80aeb333408 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:18.167546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7942bfe-b6aa-4d04-9634-d80aeb333408 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:18.169433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7942bfe-b6aa-4d04-9634-d80aeb333408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:18.176523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7942bfe-b6aa-4d04-9634-d80aeb333408 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:18.223917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 879/1757] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:18.226563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bbaa1-e52d-4899-a540-0cf0183ed2fd admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:18.238957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bbaa1-e52d-4899-a540-0cf0183ed2fd admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:18.286860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560bce8-4b8a-4863-9c0d-89bb1e86e0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:18.287638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560bce8-4b8a-4863-9c0d-89bb1e86e0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:18.287924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560bce8-4b8a-4863-9c0d-89bb1e86e0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:18.288201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560bce8-4b8a-4863-9c0d-89bb1e86e0f7 admin admin] PATH_INFO: `/v3/projects/4b54ce21c38c458bb90ef6ec41dd89f8/users/a0417a79561643a9b336d3811b1fe0fc/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:18.301336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560bce8-4b8a-4863-9c0d-89bb1e86e0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b54ce21c38c458bb90ef6ec41dd89f8, user_id=a0417a79561643a9b336d3811b1fe0fc, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:18.321037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560bce8-4b8a-4863-9c0d-89bb1e86e0f7 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:18.357825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 879/1758] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:18 2022] PUT /identity/v3/projects/4b54ce21c38c458bb90ef6ec41dd89f8/users/a0417a79561643a9b336d3811b1fe0fc/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:18.366820 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d9b24-4e31-4c36-aab1-e8c09e5a92fa None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:18.367230 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d9b24-4e31-4c36-aab1-e8c09e5a92fa None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:18.367590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d9b24-4e31-4c36-aab1-e8c09e5a92fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:18.464497 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-1b7d9b24-4e31-4c36-aab1-e8c09e5a92fa None None] MFA Rules not processed for user `a0417a79561643a9b336d3811b1fe0fc`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:18.465970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1b7d9b24-4e31-4c36-aab1-e8c09e5a92fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:18.703340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 880/1759] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:18 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:18.715847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560bce8-4b8a-4863-9c0d-89bb1e86e0f7 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:18.719349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7560bce8-4b8a-4863-9c0d-89bb1e86e0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:19.005798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560bce8-4b8a-4863-9c0d-89bb1e86e0f7 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:19.021181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36405c4b-a508-4589-9b4c-85fbadea171b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:19.022227 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36405c4b-a508-4589-9b4c-85fbadea171b service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:19.022745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36405c4b-a508-4589-9b4c-85fbadea171b service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:19.023276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36405c4b-a508-4589-9b4c-85fbadea171b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:19.060259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36405c4b-a508-4589-9b4c-85fbadea171b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:19.071796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36405c4b-a508-4589-9b4c-85fbadea171b service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:19.119610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 880/1760] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:18 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:19.300904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d9b24-4e31-4c36-aab1-e8c09e5a92fa None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:19.313826 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d9b24-4e31-4c36-aab1-e8c09e5a92fa None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:19.332896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f276100-833e-4747-8529-b291c68c9bb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:19.333585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276100-833e-4747-8529-b291c68c9bb0 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:19.336401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276100-833e-4747-8529-b291c68c9bb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:19.336682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f276100-833e-4747-8529-b291c68c9bb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:19.552259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f276100-833e-4747-8529-b291c68c9bb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:19.553997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f276100-833e-4747-8529-b291c68c9bb0 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:19.610221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 881/1761] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:19 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:19.671383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7bc99-914f-4fcd-b7ce-04e448787f5d None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:19.671927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7bc99-914f-4fcd-b7ce-04e448787f5d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:19.672384 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7bc99-914f-4fcd-b7ce-04e448787f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:19.730638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e3b7bc99-914f-4fcd-b7ce-04e448787f5d None None] MFA Rules not processed for user `05e9009410a54ce3ab88fa0cac87a7f9`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:19.736330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e3b7bc99-914f-4fcd-b7ce-04e448787f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:19.985877 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 881/1762] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Oct 3 14:52:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:20.009676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f276100-833e-4747-8529-b291c68c9bb0 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:20.011570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-0f276100-833e-4747-8529-b291c68c9bb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:20.032847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f276100-833e-4747-8529-b291c68c9bb0 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:20.045785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b59420-07be-4948-a1c3-de0c1c3bf399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:20.046569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b59420-07be-4948-a1c3-de0c1c3bf399 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:20.046875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b59420-07be-4948-a1c3-de0c1c3bf399 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:20.048361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b59420-07be-4948-a1c3-de0c1c3bf399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:20.071216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b59420-07be-4948-a1c3-de0c1c3bf399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:20.072919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b59420-07be-4948-a1c3-de0c1c3bf399 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:20.100532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 882/1763] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:20.842340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89325185-dd73-45c6-bd85-e00ba19c6300 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:20.842340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89325185-dd73-45c6-bd85-e00ba19c6300 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:20.843052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89325185-dd73-45c6-bd85-e00ba19c6300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:20.903397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-89325185-dd73-45c6-bd85-e00ba19c6300 None None] MFA Rules not processed for user `9e5325f010ae4359bd6b514a1995614d`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:20.904642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-89325185-dd73-45c6-bd85-e00ba19c6300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:21.017111 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14d9ff-210f-44ae-8318-59cceae2fc95 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:21.017622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14d9ff-210f-44ae-8318-59cceae2fc95 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:21.017729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14d9ff-210f-44ae-8318-59cceae2fc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:21.122599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 882/1764] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:21.144174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-be14d9ff-210f-44ae-8318-59cceae2fc95 None None] MFA Rules not processed for user `0be78e1abb3947a2ac5881b9fd2f6fae`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:21.155305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-be14d9ff-210f-44ae-8318-59cceae2fc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:21.253828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89325185-dd73-45c6-bd85-e00ba19c6300 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:21.253828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-89325185-dd73-45c6-bd85-e00ba19c6300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:21.268190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:21.436389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:21.476470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 883/1765] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:21.570009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89325185-dd73-45c6-bd85-e00ba19c6300 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:21.584221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:21.593122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:21.596668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:21.597166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] PATH_INFO: `/v3/users/bab817a1a0da43d0b5306fda917a227d` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:21.604093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bab817a1a0da43d0b5306fda917a227d)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:21.606842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:21.873051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bab817a1a0da43d0b5306fda917a227d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:21.885688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bab817a1a0da43d0b5306fda917a227d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:21.928397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bab817a1a0da43d0b5306fda917a227d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:22.021800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bab817a1a0da43d0b5306fda917a227d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:22.049736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bab817a1a0da43d0b5306fda917a227d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:22.068724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 883/1766] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:21 2022] DELETE /identity/v3/users/bab817a1a0da43d0b5306fda917a227d => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:22.073766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5cc1a-ea0c-4dc3-a4bf-43c34ad141af None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:22.074641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5cc1a-ea0c-4dc3-a4bf-43c34ad141af None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:22.075284 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5cc1a-ea0c-4dc3-a4bf-43c34ad141af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:22.132244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e2b5cc1a-ea0c-4dc3-a4bf-43c34ad141af None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:22.143221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e2b5cc1a-ea0c-4dc3-a4bf-43c34ad141af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:22.565615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 884/1767] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:22.596225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:22.599073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:22.813198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4aee3-65b3-4fdb-98e2-f794ae4c09c3 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:22.825077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238187c-7a92-4ddb-9563-12858dccc6e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:22.826029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238187c-7a92-4ddb-9563-12858dccc6e7 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:22.826648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238187c-7a92-4ddb-9563-12858dccc6e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:22.827074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238187c-7a92-4ddb-9563-12858dccc6e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:22.850358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238187c-7a92-4ddb-9563-12858dccc6e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:22.852871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238187c-7a92-4ddb-9563-12858dccc6e7 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:22.880054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 884/1768] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:25.812693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7758d53-c8f1-45e1-8ef6-de2a98b0d3c5 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:25.814108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7758d53-c8f1-45e1-8ef6-de2a98b0d3c5 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:25.814535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7758d53-c8f1-45e1-8ef6-de2a98b0d3c5 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:25.816279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 885/1769] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:52:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:52:25.820578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7422002-c51e-4659-9f4a-82231e6af165 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:25.821190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7422002-c51e-4659-9f4a-82231e6af165 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:25.821190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7422002-c51e-4659-9f4a-82231e6af165 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:25.823253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 885/1770] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:52:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:52:25.859409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96717f62-acd3-4952-84f7-64abbce5692a None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:25.859829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96717f62-acd3-4952-84f7-64abbce5692a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:25.860223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96717f62-acd3-4952-84f7-64abbce5692a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:25.931291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-96717f62-acd3-4952-84f7-64abbce5692a None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:25.943389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-96717f62-acd3-4952-84f7-64abbce5692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:25.991254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323787c-def9-4298-b329-9257b21c129d None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:25.992851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323787c-def9-4298-b329-9257b21c129d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:25.992851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323787c-def9-4298-b329-9257b21c129d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:26.108337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a323787c-def9-4298-b329-9257b21c129d None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:26.109752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a323787c-def9-4298-b329-9257b21c129d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:26.249895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 886/1771] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:26.252316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96717f62-acd3-4952-84f7-64abbce5692a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:26.257623 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-96717f62-acd3-4952-84f7-64abbce5692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:26.296411 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96717f62-acd3-4952-84f7-64abbce5692a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:26.310965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:26.312431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:26.312866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:26.325084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:26.329671 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] PATH_INFO: `/v3/projects/3f7c3d0e34d54538a20faae27cd5f8e4` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:26.334386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f7c3d0e34d54538a20faae27cd5f8e4)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:26.338494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:26.368010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 886/1772] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:26.448071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323787c-def9-4298-b329-9257b21c129d None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:26.449864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a323787c-def9-4298-b329-9257b21c129d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:26.503076 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323787c-def9-4298-b329-9257b21c129d None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:26.533626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f7c3d0e34d54538a20faae27cd5f8e4', 'request_id': 'req-bef344cd-8bcf-49ce-83a8-3c589e7281d2'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:26.550759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f7c3d0e34d54538a20faae27cd5f8e4', 'request_id': 'req-bef344cd-8bcf-49ce-83a8-3c589e7281d2'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:26.552485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:26.613031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f7c3d0e34d54538a20faae27cd5f8e4', 'request_id': 'req-bef344cd-8bcf-49ce-83a8-3c589e7281d2'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:26.663694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] The token cache is being invalidate because project 3f7c3d0e34d54538a20faae27cd5f8e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:26.665772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:26.667726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:26.672389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 887/1773] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:26 2022] DELETE /identity/v3/projects/3f7c3d0e34d54538a20faae27cd5f8e4 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:26.687621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:26.687621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:26.750413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:26.754319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:26.754741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:26.756202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] PATH_INFO: `/v3/users/3d230304235543449d4ca3bd297f649e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:26.759641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d230304235543449d4ca3bd297f649e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:26.763975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:26.874465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef344cd-8bcf-49ce-83a8-3c589e7281d2 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:26.883262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:26.884287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:26.885054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:26.885378 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] PATH_INFO: `/v3/users/e2c4ac1bfaa543c6b37d796a9345eb32` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:26.888388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2c4ac1bfaa543c6b37d796a9345eb32)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:26.888936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d230304235543449d4ca3bd297f649e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:26.890430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d230304235543449d4ca3bd297f649e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:26.891462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:26.902902 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d230304235543449d4ca3bd297f649e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:26.928029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d230304235543449d4ca3bd297f649e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:26.944032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-720f3474-9be1-4172-9d99-50dec20bccc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d230304235543449d4ca3bd297f649e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:26.955008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 887/1774] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:26 2022] DELETE /identity/v3/users/3d230304235543449d4ca3bd297f649e => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:26.965175 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677506b-c3a5-4e41-a76d-25705487d503 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:26.966663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677506b-c3a5-4e41-a76d-25705487d503 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:26.968179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677506b-c3a5-4e41-a76d-25705487d503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:27.015038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2c4ac1bfaa543c6b37d796a9345eb32'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:27.016042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2c4ac1bfaa543c6b37d796a9345eb32'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:27.030551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2c4ac1bfaa543c6b37d796a9345eb32'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:27.045069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b677506b-c3a5-4e41-a76d-25705487d503 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:27.049581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b677506b-c3a5-4e41-a76d-25705487d503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:27.073115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2c4ac1bfaa543c6b37d796a9345eb32'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:27.074086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2c4ac1bfaa543c6b37d796a9345eb32'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:27.117064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 888/1775] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:26 2022] DELETE /identity/v3/users/e2c4ac1bfaa543c6b37d796a9345eb32 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:27.260011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 888/1776] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:27.337134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:27.339556 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:27.517052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25f3d0-72a4-492d-96a8-2870ddcef54d admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:27.565853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7bfba-6347-43b9-8968-e961f9a12600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:27.568871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7bfba-6347-43b9-8968-e961f9a12600 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:27.569824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7bfba-6347-43b9-8968-e961f9a12600 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:27.571729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7bfba-6347-43b9-8968-e961f9a12600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:27.619033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:27.620664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677506b-c3a5-4e41-a76d-25705487d503 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:27.627750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b677506b-c3a5-4e41-a76d-25705487d503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:27.674697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:27.773096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7bfba-6347-43b9-8968-e961f9a12600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:27.778626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7bfba-6347-43b9-8968-e961f9a12600 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:27.845701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 889/1777] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:27.848922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7bfba-6347-43b9-8968-e961f9a12600 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:27.852392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-baf7bfba-6347-43b9-8968-e961f9a12600 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:27.904723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677506b-c3a5-4e41-a76d-25705487d503 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:28.103252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7bfba-6347-43b9-8968-e961f9a12600 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:28.146084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f57e36-8868-4ea5-9df9-f26e7d3e0c30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:28.146728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f57e36-8868-4ea5-9df9-f26e7d3e0c30 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:28.150083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f57e36-8868-4ea5-9df9-f26e7d3e0c30 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:28.150519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f57e36-8868-4ea5-9df9-f26e7d3e0c30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:28.152891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf78748-16c5-4060-8c70-2452dd204be7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:28.153865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf78748-16c5-4060-8c70-2452dd204be7 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:28.154149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf78748-16c5-4060-8c70-2452dd204be7 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:28.154433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf78748-16c5-4060-8c70-2452dd204be7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:28.164737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-08f57e36-8868-4ea5-9df9-f26e7d3e0c30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:28.176473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4cf78748-16c5-4060-8c70-2452dd204be7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:28.233340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f57e36-8868-4ea5-9df9-f26e7d3e0c30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:28.241455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f57e36-8868-4ea5-9df9-f26e7d3e0c30 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:28.245848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf78748-16c5-4060-8c70-2452dd204be7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:28.247566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf78748-16c5-4060-8c70-2452dd204be7 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:28.276738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 889/1778] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:28.313942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 890/1779] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:29.103394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf78748-16c5-4060-8c70-2452dd204be7 service nova] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:29.103934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4cf78748-16c5-4060-8c70-2452dd204be7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:29.145269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf78748-16c5-4060-8c70-2452dd204be7 service nova] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:29.157309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:29.158992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:29.159308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:29.160477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] PATH_INFO: `/v3/projects/c014246e603e463fbc02d7261ab9db4f` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:29.163033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c014246e603e463fbc02d7261ab9db4f)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:29.168584 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:29.169718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f57e36-8868-4ea5-9df9-f26e7d3e0c30 service nova] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:29.171653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-08f57e36-8868-4ea5-9df9-f26e7d3e0c30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:29.215218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f57e36-8868-4ea5-9df9-f26e7d3e0c30 service nova] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:29.234435 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:29.240099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:29.240567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:29.241320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] PATH_INFO: `/v3/projects/c88bb75b19f947838d04c0013ee4773d` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:29.246486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c88bb75b19f947838d04c0013ee4773d)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:29.252360 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:29.281731 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c014246e603e463fbc02d7261ab9db4f', 'request_id': 'req-3a35538b-15a4-4564-8752-ce1b77580834'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.318604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c014246e603e463fbc02d7261ab9db4f', 'request_id': 'req-3a35538b-15a4-4564-8752-ce1b77580834'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.333603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c014246e603e463fbc02d7261ab9db4f', 'request_id': 'req-3a35538b-15a4-4564-8752-ce1b77580834'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.352531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] The token cache is being invalidate because project c014246e603e463fbc02d7261ab9db4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:29.353184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.354228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.356313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 890/1780] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:29 2022] DELETE /identity/v3/projects/c014246e603e463fbc02d7261ab9db4f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:29.360720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:29.364038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:29.397482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c88bb75b19f947838d04c0013ee4773d', 'request_id': 'req-1c828c05-9a56-4272-bf1d-4037d778e42a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.408156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c88bb75b19f947838d04c0013ee4773d', 'request_id': 'req-1c828c05-9a56-4272-bf1d-4037d778e42a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.411914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:29.417546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c88bb75b19f947838d04c0013ee4773d', 'request_id': 'req-1c828c05-9a56-4272-bf1d-4037d778e42a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.431722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] The token cache is being invalidate because project c88bb75b19f947838d04c0013ee4773d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:29.432151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.432960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.434842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 891/1781] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:29 2022] DELETE /identity/v3/projects/c88bb75b19f947838d04c0013ee4773d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:29.438682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:29.440337 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:29.494018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:29.500092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35538b-15a4-4564-8752-ce1b77580834 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:29.565840 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c828c05-9a56-4272-bf1d-4037d778e42a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:29.578382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:29.579027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:29.579304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:29.581029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] PATH_INFO: `/v3/users/ddfe900ac9214cf2994a0a95ce557688` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:29.582749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddfe900ac9214cf2994a0a95ce557688)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:29.584734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:29.591133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:29.591783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:29.592068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:29.592346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] PATH_INFO: `/v3/users/a0417a79561643a9b336d3811b1fe0fc` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:29.594053 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0417a79561643a9b336d3811b1fe0fc)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:29.597620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:29.640661 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddfe900ac9214cf2994a0a95ce557688'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.642241 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddfe900ac9214cf2994a0a95ce557688'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.652116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0417a79561643a9b336d3811b1fe0fc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.652850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0417a79561643a9b336d3811b1fe0fc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.652980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddfe900ac9214cf2994a0a95ce557688'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.661540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0417a79561643a9b336d3811b1fe0fc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.665758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddfe900ac9214cf2994a0a95ce557688'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.666474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddfe900ac9214cf2994a0a95ce557688'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.674983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0417a79561643a9b336d3811b1fe0fc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.684705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0417a79561643a9b336d3811b1fe0fc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:29.690655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 892/1782] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:29 2022] DELETE /identity/v3/users/ddfe900ac9214cf2994a0a95ce557688 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:29.695019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 891/1783] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:29 2022] DELETE /identity/v3/users/a0417a79561643a9b336d3811b1fe0fc => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:29.980268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:29.985709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:29.985981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:29.985981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:30.056676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:30.057341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012035f9-0222-4174-8d2e-43b0b8a4c614 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.072366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684be26e-315b-4014-b7b3-fad0bfc04da2 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.078884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.079520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.079848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.080122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] PATH_INFO: `/v3/projects/398371e95a9e43aa81696a6a76624f66` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.081288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=398371e95a9e43aa81696a6a76624f66)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.083774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.106362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:30.125269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.127344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.127344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.127344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] PATH_INFO: `/v3/projects/4b54ce21c38c458bb90ef6ec41dd89f8` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.127758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b54ce21c38c458bb90ef6ec41dd89f8)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.130328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.139504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '398371e95a9e43aa81696a6a76624f66', 'request_id': 'req-96437d87-578d-491f-ae31-722a2e4debbf'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:30.150892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '398371e95a9e43aa81696a6a76624f66', 'request_id': 'req-96437d87-578d-491f-ae31-722a2e4debbf'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:30.158018 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '398371e95a9e43aa81696a6a76624f66', 'request_id': 'req-96437d87-578d-491f-ae31-722a2e4debbf'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:30.172632 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] The token cache is being invalidate because project 398371e95a9e43aa81696a6a76624f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:30.173088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:30.174112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-96437d87-578d-491f-ae31-722a2e4debbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:30.176574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 892/1784] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:29 2022] DELETE /identity/v3/projects/398371e95a9e43aa81696a6a76624f66 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:30.182542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee05e-457c-4a1d-821d-3648da4a5b22 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.182863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b54ce21c38c458bb90ef6ec41dd89f8', 'request_id': 'req-b8100cf5-39ee-4d94-90ba-0760ea83c548'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:30.183327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee05e-457c-4a1d-821d-3648da4a5b22 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.183327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee05e-457c-4a1d-821d-3648da4a5b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.191903 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b54ce21c38c458bb90ef6ec41dd89f8', 'request_id': 'req-b8100cf5-39ee-4d94-90ba-0760ea83c548'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:30.199488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b54ce21c38c458bb90ef6ec41dd89f8', 'request_id': 'req-b8100cf5-39ee-4d94-90ba-0760ea83c548'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:30.206248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-062ee05e-457c-4a1d-821d-3648da4a5b22 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:30.207508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-062ee05e-457c-4a1d-821d-3648da4a5b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:30.210766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] The token cache is being invalidate because project 4b54ce21c38c458bb90ef6ec41dd89f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:30.211055 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:30.211716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b8100cf5-39ee-4d94-90ba-0760ea83c548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:30.213216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 893/1785] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:29 2022] DELETE /identity/v3/projects/4b54ce21c38c458bb90ef6ec41dd89f8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:30.217907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb571bc9-bcba-4af3-9c80-b52b0c44b370 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.218260 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb571bc9-bcba-4af3-9c80-b52b0c44b370 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.218804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb571bc9-bcba-4af3-9c80-b52b0c44b370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.238723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-fb571bc9-bcba-4af3-9c80-b52b0c44b370 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:30.239941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-fb571bc9-bcba-4af3-9c80-b52b0c44b370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:30.292209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 893/1786] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:30.300678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb70e-cf7c-481f-b975-0b430fd6b67b None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.301090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb70e-cf7c-481f-b975-0b430fd6b67b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.301404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb70e-cf7c-481f-b975-0b430fd6b67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.321543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 894/1787] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:30.322177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9d4fb70e-cf7c-481f-b975-0b430fd6b67b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:30.323384 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9d4fb70e-cf7c-481f-b975-0b430fd6b67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:30.329011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cea6a-628c-4d8c-80e1-4aa4acd835c1 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.329302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cea6a-628c-4d8c-80e1-4aa4acd835c1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.329791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cea6a-628c-4d8c-80e1-4aa4acd835c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.332484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 894/1788] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:30.336363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fb70e-cf7c-481f-b975-0b430fd6b67b None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.350132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fb70e-cf7c-481f-b975-0b430fd6b67b None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.356762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade9356-7906-4115-990f-162621247bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.357194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9356-7906-4115-990f-162621247bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.357594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9356-7906-4115-990f-162621247bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.357769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9356-7906-4115-990f-162621247bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.358429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade9356-7906-4115-990f-162621247bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:52:30.358758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade9356-7906-4115-990f-162621247bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.358939 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e50cea6a-628c-4d8c-80e1-4aa4acd835c1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:30.360324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade9356-7906-4115-990f-162621247bf6 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.361705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e50cea6a-628c-4d8c-80e1-4aa4acd835c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:30.375979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 895/1789] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:30.376145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 895/1790] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:52:30.377068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cea6a-628c-4d8c-80e1-4aa4acd835c1 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.382782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade9356-7906-4115-990f-162621247bf6 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.387459 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cea6a-628c-4d8c-80e1-4aa4acd835c1 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.392236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea610-baf9-4d85-b6bc-dd288a7db09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.392850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea610-baf9-4d85-b6bc-dd288a7db09e admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.393135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea610-baf9-4d85-b6bc-dd288a7db09e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.393415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea610-baf9-4d85-b6bc-dd288a7db09e admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.394293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bea610-baf9-4d85-b6bc-dd288a7db09e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:52:30.394600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bea610-baf9-4d85-b6bc-dd288a7db09e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.396207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bea610-baf9-4d85-b6bc-dd288a7db09e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.407821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade9356-7906-4115-990f-162621247bf6 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.408769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 896/1791] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:52:30.412751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea610-baf9-4d85-b6bc-dd288a7db09e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.420570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f65e5f-80dc-4aa8-b88b-68584c650cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.422005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f65e5f-80dc-4aa8-b88b-68584c650cba admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.422431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f65e5f-80dc-4aa8-b88b-68584c650cba admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.422961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f65e5f-80dc-4aa8-b88b-68584c650cba admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.423261 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea610-baf9-4d85-b6bc-dd288a7db09e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.424006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f65e5f-80dc-4aa8-b88b-68584c650cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.428563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f65e5f-80dc-4aa8-b88b-68584c650cba admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.429360 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c258342-1a4b-4bf8-a41e-7c98c8bf775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.430004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c258342-1a4b-4bf8-a41e-7c98c8bf775c admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.430275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c258342-1a4b-4bf8-a41e-7c98c8bf775c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.430548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c258342-1a4b-4bf8-a41e-7c98c8bf775c admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.431308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c258342-1a4b-4bf8-a41e-7c98c8bf775c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.433235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c258342-1a4b-4bf8-a41e-7c98c8bf775c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.480527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 896/1792] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:30.480705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 897/1793] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:52:30.481705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f65e5f-80dc-4aa8-b88b-68584c650cba admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.482968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c258342-1a4b-4bf8-a41e-7c98c8bf775c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.529110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:30.533614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:30.559581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f65e5f-80dc-4aa8-b88b-68584c650cba admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.564830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d634-2b02-423e-b47f-1825fc456ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.566314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d634-2b02-423e-b47f-1825fc456ddc admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.566449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d634-2b02-423e-b47f-1825fc456ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.566758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d634-2b02-423e-b47f-1825fc456ddc admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.567363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7d634-2b02-423e-b47f-1825fc456ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.570080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7d634-2b02-423e-b47f-1825fc456ddc admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.572997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c258342-1a4b-4bf8-a41e-7c98c8bf775c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.578935 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd499483-0f79-4f55-980d-3f5f1ea120e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.579622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd499483-0f79-4f55-980d-3f5f1ea120e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.579944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd499483-0f79-4f55-980d-3f5f1ea120e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.581041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd499483-0f79-4f55-980d-3f5f1ea120e8 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.582413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd499483-0f79-4f55-980d-3f5f1ea120e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.585001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd499483-0f79-4f55-980d-3f5f1ea120e8 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.602726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 897/1794] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:30.607344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d634-2b02-423e-b47f-1825fc456ddc admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.617999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d634-2b02-423e-b47f-1825fc456ddc admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.623116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd4707-4355-4aaa-8d07-8c0df384948a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.623887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4707-4355-4aaa-8d07-8c0df384948a admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.624445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4707-4355-4aaa-8d07-8c0df384948a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.624869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4707-4355-4aaa-8d07-8c0df384948a admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.625725 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd4707-4355-4aaa-8d07-8c0df384948a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.627137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 898/1795] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/users => generated 290 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:52:30.627671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd4707-4355-4aaa-8d07-8c0df384948a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.631621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd499483-0f79-4f55-980d-3f5f1ea120e8 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.639336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 898/1796] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:30.642106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd499483-0f79-4f55-980d-3f5f1ea120e8 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.643162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd4707-4355-4aaa-8d07-8c0df384948a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.651933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25b4b-cc91-4fb2-ad37-5b74935f2393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.652552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25b4b-cc91-4fb2-ad37-5b74935f2393 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.652832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25b4b-cc91-4fb2-ad37-5b74935f2393 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.653101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25b4b-cc91-4fb2-ad37-5b74935f2393 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.653771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b25b4b-cc91-4fb2-ad37-5b74935f2393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.654687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd4707-4355-4aaa-8d07-8c0df384948a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.655439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b25b4b-cc91-4fb2-ad37-5b74935f2393 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.661905 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14b18a-735c-43b6-bf9d-7f8e43518085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.662544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14b18a-735c-43b6-bf9d-7f8e43518085 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.662823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14b18a-735c-43b6-bf9d-7f8e43518085 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.663097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14b18a-735c-43b6-bf9d-7f8e43518085 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.663753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14b18a-735c-43b6-bf9d-7f8e43518085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.665349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14b18a-735c-43b6-bf9d-7f8e43518085 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.666666 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 899/1797] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:30.670518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25b4b-cc91-4fb2-ad37-5b74935f2393 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.679039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 899/1798] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:30.682899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14b18a-735c-43b6-bf9d-7f8e43518085 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.683437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25b4b-cc91-4fb2-ad37-5b74935f2393 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.688780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfe366-9b74-4653-b4c8-bbdea2e201eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.689968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfe366-9b74-4653-b4c8-bbdea2e201eb admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.691444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfe366-9b74-4653-b4c8-bbdea2e201eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.691444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfe366-9b74-4653-b4c8-bbdea2e201eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.691690 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfe366-9b74-4653-b4c8-bbdea2e201eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.694037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14b18a-735c-43b6-bf9d-7f8e43518085 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.694157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfe366-9b74-4653-b4c8-bbdea2e201eb admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.698981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f200e-b9a6-44ec-bce7-7671dd2e5e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.700410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f200e-b9a6-44ec-bce7-7671dd2e5e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.700410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f200e-b9a6-44ec-bce7-7671dd2e5e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.700600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f200e-b9a6-44ec-bce7-7671dd2e5e09 admin admin] PATH_INFO: `/v3/projects/faa01410bcd540238d89661d37909e6f/users/9b070b64791c4803ab705478dda87534/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.707208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 900/1799] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:30.713900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfe366-9b74-4653-b4c8-bbdea2e201eb admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.719239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f200e-b9a6-44ec-bce7-7671dd2e5e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa01410bcd540238d89661d37909e6f, user_id=9b070b64791c4803ab705478dda87534, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.723321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f200e-b9a6-44ec-bce7-7671dd2e5e09 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.725155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfe366-9b74-4653-b4c8-bbdea2e201eb admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.730484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d98c0-b999-4b17-b376-87210a9b0857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.731222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d98c0-b999-4b17-b376-87210a9b0857 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.731523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d98c0-b999-4b17-b376-87210a9b0857 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.731895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d98c0-b999-4b17-b376-87210a9b0857 admin admin] PATH_INFO: `/v3/projects/e5cb00920a7e4291b6ee8dd70e4d6026/users/939dccdf404741eaacb60c298f0d9c7f/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.743235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 900/1800] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:30 2022] PUT /identity/v3/projects/faa01410bcd540238d89661d37909e6f/users/9b070b64791c4803ab705478dda87534/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:30.756294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910743d-6ee9-4e53-a7ed-abb6dedeba2c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.756727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910743d-6ee9-4e53-a7ed-abb6dedeba2c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.756916 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910743d-6ee9-4e53-a7ed-abb6dedeba2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.763771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d98c0-b999-4b17-b376-87210a9b0857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5cb00920a7e4291b6ee8dd70e4d6026, user_id=939dccdf404741eaacb60c298f0d9c7f, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.767149 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d98c0-b999-4b17-b376-87210a9b0857 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.785800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 901/1801] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:30 2022] PUT /identity/v3/projects/e5cb00920a7e4291b6ee8dd70e4d6026/users/939dccdf404741eaacb60c298f0d9c7f/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:30.791404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-1910743d-6ee9-4e53-a7ed-abb6dedeba2c None None] MFA Rules not processed for user `9b070b64791c4803ab705478dda87534`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:30.792682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1910743d-6ee9-4e53-a7ed-abb6dedeba2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:30.795337 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5c657-8514-4720-bf0e-cdcd148205d2 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.795624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5c657-8514-4720-bf0e-cdcd148205d2 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.795896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5c657-8514-4720-bf0e-cdcd148205d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.824742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-a5f5c657-8514-4720-bf0e-cdcd148205d2 None None] MFA Rules not processed for user `939dccdf404741eaacb60c298f0d9c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:30.825972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a5f5c657-8514-4720-bf0e-cdcd148205d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:30.859654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 901/1802] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:30.864129 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910743d-6ee9-4e53-a7ed-abb6dedeba2c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.894834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 902/1803] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:30.899435 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5c657-8514-4720-bf0e-cdcd148205d2 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.925733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910743d-6ee9-4e53-a7ed-abb6dedeba2c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.930387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276046b8-dc1e-4eb5-bf12-3fb0055791b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:30.930984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276046b8-dc1e-4eb5-bf12-3fb0055791b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:30.931262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276046b8-dc1e-4eb5-bf12-3fb0055791b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:30.931658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276046b8-dc1e-4eb5-bf12-3fb0055791b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:30.932290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276046b8-dc1e-4eb5-bf12-3fb0055791b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:30.934229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276046b8-dc1e-4eb5-bf12-3fb0055791b2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:30.955658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:30.964012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 902/1804] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:52:30.964499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5c657-8514-4720-bf0e-cdcd148205d2 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:30.968129 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276046b8-dc1e-4eb5-bf12-3fb0055791b2 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:30.976668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:31.023925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3ec27-b148-4bef-9308-fca2da773d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.024559 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3ec27-b148-4bef-9308-fca2da773d86 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.024824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3ec27-b148-4bef-9308-fca2da773d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.025097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3ec27-b148-4bef-9308-fca2da773d86 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.025774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3ec27-b148-4bef-9308-fca2da773d86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.027702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3ec27-b148-4bef-9308-fca2da773d86 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.032261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276046b8-dc1e-4eb5-bf12-3fb0055791b2 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.036886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f7049-44e4-4108-becc-f16b73970de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.037490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f7049-44e4-4108-becc-f16b73970de5 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.037789 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f7049-44e4-4108-becc-f16b73970de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.038058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f7049-44e4-4108-becc-f16b73970de5 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.038684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f7049-44e4-4108-becc-f16b73970de5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.040720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f7049-44e4-4108-becc-f16b73970de5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.054404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 903/1805] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:52:31.058516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3ec27-b148-4bef-9308-fca2da773d86 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.065018 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 903/1806] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:30 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:52:31.069375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f7049-44e4-4108-becc-f16b73970de5 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.079034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f7049-44e4-4108-becc-f16b73970de5 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.124449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3ec27-b148-4bef-9308-fca2da773d86 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.129701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f803a49-fa09-46d6-81b1-d4f5b679907a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.130470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f803a49-fa09-46d6-81b1-d4f5b679907a admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.130753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f803a49-fa09-46d6-81b1-d4f5b679907a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.131026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f803a49-fa09-46d6-81b1-d4f5b679907a admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.131651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f803a49-fa09-46d6-81b1-d4f5b679907a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.133597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f803a49-fa09-46d6-81b1-d4f5b679907a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.139422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ebe3-36d4-4a34-a58b-52ef2401a3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.140030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ebe3-36d4-4a34-a58b-52ef2401a3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.140560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ebe3-36d4-4a34-a58b-52ef2401a3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.140560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ebe3-36d4-4a34-a58b-52ef2401a3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.141326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284ebe3-36d4-4a34-a58b-52ef2401a3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.142953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284ebe3-36d4-4a34-a58b-52ef2401a3f0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.153295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 904/1807] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:31.154879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 904/1808] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:31 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:31.157666 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ebe3-36d4-4a34-a58b-52ef2401a3f0 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.158608 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f803a49-fa09-46d6-81b1-d4f5b679907a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.168023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ebe3-36d4-4a34-a58b-52ef2401a3f0 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.168311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f803a49-fa09-46d6-81b1-d4f5b679907a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.172815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24b72d-80a2-4381-93af-2c89344c294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.172938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a910dd-09a3-4d8c-b939-c82b78ee22a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.173394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24b72d-80a2-4381-93af-2c89344c294a admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.173612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a910dd-09a3-4d8c-b939-c82b78ee22a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.173708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24b72d-80a2-4381-93af-2c89344c294a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.173898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a910dd-09a3-4d8c-b939-c82b78ee22a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.173984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24b72d-80a2-4381-93af-2c89344c294a admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.174174 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a910dd-09a3-4d8c-b939-c82b78ee22a8 admin admin] PATH_INFO: `/v3/projects/ba21d715c13f47f788b4be9d0f439cef/users/79359574f0624a69984fc20d9222f16f/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.174568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24b72d-80a2-4381-93af-2c89344c294a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.176793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24b72d-80a2-4381-93af-2c89344c294a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.186438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 905/1809] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:31.189682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a910dd-09a3-4d8c-b939-c82b78ee22a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba21d715c13f47f788b4be9d0f439cef, user_id=79359574f0624a69984fc20d9222f16f, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.190422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24b72d-80a2-4381-93af-2c89344c294a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.193004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a910dd-09a3-4d8c-b939-c82b78ee22a8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.201068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24b72d-80a2-4381-93af-2c89344c294a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.206671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 905/1810] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:31 2022] PUT /identity/v3/projects/ba21d715c13f47f788b4be9d0f439cef/users/79359574f0624a69984fc20d9222f16f/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:31.210506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a910dd-09a3-4d8c-b939-c82b78ee22a8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.267501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba4288-0795-4717-8bf8-18feec4e9a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.268461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba4288-0795-4717-8bf8-18feec4e9a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.268906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba4288-0795-4717-8bf8-18feec4e9a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.269316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba4288-0795-4717-8bf8-18feec4e9a72 admin admin] PATH_INFO: `/v3/projects/f9d0841fd6a74571af7ebd65101ea53a/users/f5515e02510846fe955703282e6cdc3e/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.273164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a910dd-09a3-4d8c-b939-c82b78ee22a8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.283685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44ae9-d47d-436a-83c2-ee649bb7d126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.284827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44ae9-d47d-436a-83c2-ee649bb7d126 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.285116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44ae9-d47d-436a-83c2-ee649bb7d126 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.285388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44ae9-d47d-436a-83c2-ee649bb7d126 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.286250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44ae9-d47d-436a-83c2-ee649bb7d126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.287371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba4288-0795-4717-8bf8-18feec4e9a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d0841fd6a74571af7ebd65101ea53a, user_id=f5515e02510846fe955703282e6cdc3e, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.288017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44ae9-d47d-436a-83c2-ee649bb7d126 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.289721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:31.290135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba4288-0795-4717-8bf8-18feec4e9a72 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.306370 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 906/1811] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:31.307476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44ae9-d47d-436a-83c2-ee649bb7d126 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.308111 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 906/1812] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:31 2022] PUT /identity/v3/projects/f9d0841fd6a74571af7ebd65101ea53a/users/f5515e02510846fe955703282e6cdc3e/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:31.311959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba4288-0795-4717-8bf8-18feec4e9a72 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.318578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:31.321948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44ae9-d47d-436a-83c2-ee649bb7d126 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.387161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba4288-0795-4717-8bf8-18feec4e9a72 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.391598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12470e83-780a-40ac-b3b4-c9f3a6eb47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.392429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12470e83-780a-40ac-b3b4-c9f3a6eb47cd admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.392544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6f040-62f4-4414-aac2-1f6179509c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.392803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12470e83-780a-40ac-b3b4-c9f3a6eb47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.393041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6f040-62f4-4414-aac2-1f6179509c4e admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.393144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12470e83-780a-40ac-b3b4-c9f3a6eb47cd admin admin] PATH_INFO: `/v3/projects/ba21d715c13f47f788b4be9d0f439cef/users/79359574f0624a69984fc20d9222f16f/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.393427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6f040-62f4-4414-aac2-1f6179509c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.393766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6f040-62f4-4414-aac2-1f6179509c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.394418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6f040-62f4-4414-aac2-1f6179509c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.395442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12470e83-780a-40ac-b3b4-c9f3a6eb47cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba21d715c13f47f788b4be9d0f439cef, user_id=79359574f0624a69984fc20d9222f16f, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.396277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6f040-62f4-4414-aac2-1f6179509c4e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.398566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12470e83-780a-40ac-b3b4-c9f3a6eb47cd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.406932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 907/1813] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:31.411370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6f040-62f4-4414-aac2-1f6179509c4e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.416665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 907/1814] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:31 2022] PUT /identity/v3/projects/ba21d715c13f47f788b4be9d0f439cef/users/79359574f0624a69984fc20d9222f16f/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:31.422107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6f040-62f4-4414-aac2-1f6179509c4e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.427218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8262a2-deaa-4f92-b918-21de6ea45fb5 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.427346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8262a2-deaa-4f92-b918-21de6ea45fb5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.427565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8262a2-deaa-4f92-b918-21de6ea45fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.457692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-af8262a2-deaa-4f92-b918-21de6ea45fb5 None None] MFA Rules not processed for user `79359574f0624a69984fc20d9222f16f`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:31.458923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-af8262a2-deaa-4f92-b918-21de6ea45fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:31.499686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabcd3a-d152-42eb-98e4-8064b92f5db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.500354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabcd3a-d152-42eb-98e4-8064b92f5db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.500632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabcd3a-d152-42eb-98e4-8064b92f5db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.500910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabcd3a-d152-42eb-98e4-8064b92f5db5 admin admin] PATH_INFO: `/v3/projects/f9d0841fd6a74571af7ebd65101ea53a/users/f5515e02510846fe955703282e6cdc3e/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.502979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabcd3a-d152-42eb-98e4-8064b92f5db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d0841fd6a74571af7ebd65101ea53a, user_id=f5515e02510846fe955703282e6cdc3e, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.506096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabcd3a-d152-42eb-98e4-8064b92f5db5 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.518897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 908/1815] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:31 2022] PUT /identity/v3/projects/f9d0841fd6a74571af7ebd65101ea53a/users/f5515e02510846fe955703282e6cdc3e/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:31.528891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e3881-44cc-4848-80fe-f582d62bafa5 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.529179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e3881-44cc-4848-80fe-f582d62bafa5 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.529456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e3881-44cc-4848-80fe-f582d62bafa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.556594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 908/1816] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:31.568290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-6f3e3881-44cc-4848-80fe-f582d62bafa5 None None] MFA Rules not processed for user `f5515e02510846fe955703282e6cdc3e`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:31.569728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6f3e3881-44cc-4848-80fe-f582d62bafa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:31.571590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8262a2-deaa-4f92-b918-21de6ea45fb5 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.572995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-af8262a2-deaa-4f92-b918-21de6ea45fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:31.647856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8262a2-deaa-4f92-b918-21de6ea45fb5 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.678317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 909/1817] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:31.689806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e3881-44cc-4848-80fe-f582d62bafa5 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.691558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6f3e3881-44cc-4848-80fe-f582d62bafa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:31.691815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:31.732759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:31.741303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba734fb1-7e61-4f26-a5ea-8dc416401750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.742385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba734fb1-7e61-4f26-a5ea-8dc416401750 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.742726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba734fb1-7e61-4f26-a5ea-8dc416401750 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.743013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba734fb1-7e61-4f26-a5ea-8dc416401750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.772490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e3881-44cc-4848-80fe-f582d62bafa5 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.776095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fce586-4364-46ad-94f9-87264629997d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.776707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fce586-4364-46ad-94f9-87264629997d service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.776991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fce586-4364-46ad-94f9-87264629997d service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.777263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fce586-4364-46ad-94f9-87264629997d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.786572 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fce586-4364-46ad-94f9-87264629997d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.788008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fce586-4364-46ad-94f9-87264629997d service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.797648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 910/1818] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:52:31.805810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba734fb1-7e61-4f26-a5ea-8dc416401750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:31.807264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba734fb1-7e61-4f26-a5ea-8dc416401750 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:31.818137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 909/1819] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:52:31.934236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fce586-4364-46ad-94f9-87264629997d service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.952562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fce586-4364-46ad-94f9-87264629997d service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.957419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba734fb1-7e61-4f26-a5ea-8dc416401750 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:31.958013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eb597-bd24-4aa1-86a2-a32b4f069090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.958697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb597-bd24-4aa1-86a2-a32b4f069090 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.959046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb597-bd24-4aa1-86a2-a32b4f069090 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.959454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb597-bd24-4aa1-86a2-a32b4f069090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:31.967752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba734fb1-7e61-4f26-a5ea-8dc416401750 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:31.972024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6da644-67ae-4834-af26-74552da1626d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:31.972638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6da644-67ae-4834-af26-74552da1626d service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:31.972920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6da644-67ae-4834-af26-74552da1626d service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:31.973197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6da644-67ae-4834-af26-74552da1626d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:32.035564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6da644-67ae-4834-af26-74552da1626d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:32.037012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6da644-67ae-4834-af26-74552da1626d service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:32.048681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 910/1820] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:52:32.051191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8eb597-bd24-4aa1-86a2-a32b4f069090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:32.052672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8eb597-bd24-4aa1-86a2-a32b4f069090 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:32.063062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 911/1821] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:32.335965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6da644-67ae-4834-af26-74552da1626d service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:32.404176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6da644-67ae-4834-af26-74552da1626d service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:32.408980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:32.410246 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:32.410526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:32.410812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] PATH_INFO: `/v3/users/9b070b64791c4803ab705478dda87534` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:32.411921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b070b64791c4803ab705478dda87534)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:32.413925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:32.435453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:32.459672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b070b64791c4803ab705478dda87534'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:32.460339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b070b64791c4803ab705478dda87534'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:32.467140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b070b64791c4803ab705478dda87534'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:32.479653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b070b64791c4803ab705478dda87534'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:32.487179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b070b64791c4803ab705478dda87534'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:32.493471 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 911/1822] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:32 2022] DELETE /identity/v3/users/9b070b64791c4803ab705478dda87534 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:32.497774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead99cd7-8f04-44a9-ad0f-3f075023a11b None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:32.498270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead99cd7-8f04-44a9-ad0f-3f075023a11b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:32.498358 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead99cd7-8f04-44a9-ad0f-3f075023a11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:32.517899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ead99cd7-8f04-44a9-ad0f-3f075023a11b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:32.519616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ead99cd7-8f04-44a9-ad0f-3f075023a11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:32.580638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 912/1823] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:32.590937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:32.592355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:32.653776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b081dd-c56e-4dbf-9b9a-7deb9dc7cc5b admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:32.660308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2309c-8b08-465f-ad7f-d0c75a740fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:32.661011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2309c-8b08-465f-ad7f-d0c75a740fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:32.661368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2309c-8b08-465f-ad7f-d0c75a740fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:32.661743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2309c-8b08-465f-ad7f-d0c75a740fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:32.671365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2309c-8b08-465f-ad7f-d0c75a740fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:32.672979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2309c-8b08-465f-ad7f-d0c75a740fc6 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:32.682531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 912/1824] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:32.984854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead99cd7-8f04-44a9-ad0f-3f075023a11b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:32.986428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ead99cd7-8f04-44a9-ad0f-3f075023a11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:33.001185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead99cd7-8f04-44a9-ad0f-3f075023a11b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:33.007014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:33.007737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:33.008235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:33.008480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] PATH_INFO: `/v3/projects/faa01410bcd540238d89661d37909e6f` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:33.009741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa01410bcd540238d89661d37909e6f)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:33.012311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:33.062369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa01410bcd540238d89661d37909e6f', 'request_id': 'req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.070491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa01410bcd540238d89661d37909e6f', 'request_id': 'req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.071685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:33.077743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa01410bcd540238d89661d37909e6f', 'request_id': 'req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.091511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] The token cache is being invalidate because project faa01410bcd540238d89661d37909e6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:33.091796 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.092454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.094223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 913/1825] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:32 2022] DELETE /identity/v3/projects/faa01410bcd540238d89661d37909e6f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:33.098565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2309c-8b08-465f-ad7f-d0c75a740fc6 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:33.108704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2309c-8b08-465f-ad7f-d0c75a740fc6 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:33.167079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:33.167698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:33.167982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:33.168255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] PATH_INFO: `/v3/users/79359574f0624a69984fc20d9222f16f` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:33.169371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79359574f0624a69984fc20d9222f16f)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:33.171330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:33.215781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79359574f0624a69984fc20d9222f16f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.216629 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79359574f0624a69984fc20d9222f16f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.222298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:33.224372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79359574f0624a69984fc20d9222f16f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.238951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79359574f0624a69984fc20d9222f16f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.247132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-77de5231-10e5-4604-865d-2dd6fbd8ead2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79359574f0624a69984fc20d9222f16f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.253270 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 913/1826] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:33 2022] DELETE /identity/v3/users/79359574f0624a69984fc20d9222f16f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:33.522552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:33.524043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:33.537786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243cdc1-9c6f-4bc5-96fd-02893f7dcc85 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:33.601709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:33.602371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:33.602676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:33.602944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] PATH_INFO: `/v3/projects/ba21d715c13f47f788b4be9d0f439cef` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:33.604201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba21d715c13f47f788b4be9d0f439cef)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:33.606261 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:33.660651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba21d715c13f47f788b4be9d0f439cef', 'request_id': 'req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.668683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba21d715c13f47f788b4be9d0f439cef', 'request_id': 'req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.677669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba21d715c13f47f788b4be9d0f439cef', 'request_id': 'req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.689194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] The token cache is being invalidate because project ba21d715c13f47f788b4be9d0f439cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:33.689537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.690169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ce9bbdba-9147-4b2f-9fc4-cc13baf1fcb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:33.696777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 914/1827] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:33 2022] DELETE /identity/v3/projects/ba21d715c13f47f788b4be9d0f439cef => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:33.698277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd015e-168c-48fc-86bf-694e53e9c9ca None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:33.698761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd015e-168c-48fc-86bf-694e53e9c9ca None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:33.698761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd015e-168c-48fc-86bf-694e53e9c9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:33.718183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f5cd015e-168c-48fc-86bf-694e53e9c9ca None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:33.719910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f5cd015e-168c-48fc-86bf-694e53e9c9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:33.789539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 914/1828] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:33.796315 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668d076-6b71-46b7-b5f5-336a0186fedd None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:33.796580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668d076-6b71-46b7-b5f5-336a0186fedd None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:33.797046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668d076-6b71-46b7-b5f5-336a0186fedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:33.825374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c668d076-6b71-46b7-b5f5-336a0186fedd None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:33.826616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c668d076-6b71-46b7-b5f5-336a0186fedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:33.836951 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 915/1829] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:33.846530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd015e-168c-48fc-86bf-694e53e9c9ca None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:33.858171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd015e-168c-48fc-86bf-694e53e9c9ca None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:33.863444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d5cb2-cf8e-4812-9cf0-c669f535a98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:33.864073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d5cb2-cf8e-4812-9cf0-c669f535a98c admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:33.864358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d5cb2-cf8e-4812-9cf0-c669f535a98c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:33.864626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d5cb2-cf8e-4812-9cf0-c669f535a98c admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:33.865275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d5cb2-cf8e-4812-9cf0-c669f535a98c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:52:33.865606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d5cb2-cf8e-4812-9cf0-c669f535a98c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:33.867178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d5cb2-cf8e-4812-9cf0-c669f535a98c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:33.879414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 915/1830] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:52:33.883857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668d076-6b71-46b7-b5f5-336a0186fedd None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:33.894129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668d076-6b71-46b7-b5f5-336a0186fedd None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:33.898997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb06d4e-fd40-4f56-84e7-6a8d1509ba13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:33.899601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb06d4e-fd40-4f56-84e7-6a8d1509ba13 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:33.899876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb06d4e-fd40-4f56-84e7-6a8d1509ba13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:33.900189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb06d4e-fd40-4f56-84e7-6a8d1509ba13 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:33.900837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb06d4e-fd40-4f56-84e7-6a8d1509ba13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:33.902767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb06d4e-fd40-4f56-84e7-6a8d1509ba13 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:33.935724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 916/1831] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:33 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:33.942842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d5cb2-cf8e-4812-9cf0-c669f535a98c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:33.953707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d5cb2-cf8e-4812-9cf0-c669f535a98c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:34.012021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f708478-9dc8-4de2-a59c-dba7a8492196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:34.012643 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f708478-9dc8-4de2-a59c-dba7a8492196 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:34.012925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f708478-9dc8-4de2-a59c-dba7a8492196 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:34.013201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f708478-9dc8-4de2-a59c-dba7a8492196 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:34.013910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f708478-9dc8-4de2-a59c-dba7a8492196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:34.015870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f708478-9dc8-4de2-a59c-dba7a8492196 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:34.041430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 916/1832] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:33 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:34.042673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb06d4e-fd40-4f56-84e7-6a8d1509ba13 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:34.053413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb06d4e-fd40-4f56-84e7-6a8d1509ba13 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:34.058351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6c24d-9f8f-4f9d-bac6-0bb58000f827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:34.059585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6c24d-9f8f-4f9d-bac6-0bb58000f827 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:34.060003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6c24d-9f8f-4f9d-bac6-0bb58000f827 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:34.060268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6c24d-9f8f-4f9d-bac6-0bb58000f827 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:34.060910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6c24d-9f8f-4f9d-bac6-0bb58000f827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:34.062597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6c24d-9f8f-4f9d-bac6-0bb58000f827 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:34.075597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 917/1833] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:34.076997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f708478-9dc8-4de2-a59c-dba7a8492196 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:34.087625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f708478-9dc8-4de2-a59c-dba7a8492196 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:34.092645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d3328-ef44-4515-a7c9-4b40de33e43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:34.093778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d3328-ef44-4515-a7c9-4b40de33e43c admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:34.093980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d3328-ef44-4515-a7c9-4b40de33e43c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:34.094253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d3328-ef44-4515-a7c9-4b40de33e43c admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:34.095087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d3328-ef44-4515-a7c9-4b40de33e43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:34.097491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d3328-ef44-4515-a7c9-4b40de33e43c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:34.112719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 917/1834] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:34.113904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6c24d-9f8f-4f9d-bac6-0bb58000f827 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:34.123807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6c24d-9f8f-4f9d-bac6-0bb58000f827 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:34.128727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73da29-1d7b-4f06-8104-fc9ea9d40e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:34.129429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73da29-1d7b-4f06-8104-fc9ea9d40e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:34.129819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73da29-1d7b-4f06-8104-fc9ea9d40e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:34.130104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73da29-1d7b-4f06-8104-fc9ea9d40e6c admin admin] PATH_INFO: `/v3/projects/f9a7e11e3b334c7faf028d8a297f726e/users/2f60a2965fe4432592f392f986857bee/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:34.145453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73da29-1d7b-4f06-8104-fc9ea9d40e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a7e11e3b334c7faf028d8a297f726e, user_id=2f60a2965fe4432592f392f986857bee, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:34.148114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73da29-1d7b-4f06-8104-fc9ea9d40e6c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:34.160776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 918/1835] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:34 2022] PUT /identity/v3/projects/f9a7e11e3b334c7faf028d8a297f726e/users/2f60a2965fe4432592f392f986857bee/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:34.171795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379d6f7-9485-47e8-9de5-cf9c553111a7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:34.172178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379d6f7-9485-47e8-9de5-cf9c553111a7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:34.172520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379d6f7-9485-47e8-9de5-cf9c553111a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:34.200251 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e379d6f7-9485-47e8-9de5-cf9c553111a7 None None] MFA Rules not processed for user `2f60a2965fe4432592f392f986857bee`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:34.201482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e379d6f7-9485-47e8-9de5-cf9c553111a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:34.215299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:34.271042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 918/1836] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:34.282356 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73da29-1d7b-4f06-8104-fc9ea9d40e6c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:34.284099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6f73da29-1d7b-4f06-8104-fc9ea9d40e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:34.302375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:34.351647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73da29-1d7b-4f06-8104-fc9ea9d40e6c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:34.355614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53761838-bc32-46ff-8623-daf47a5963c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:34.356259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53761838-bc32-46ff-8623-daf47a5963c0 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:34.356585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53761838-bc32-46ff-8623-daf47a5963c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:34.356851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53761838-bc32-46ff-8623-daf47a5963c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:34.366008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53761838-bc32-46ff-8623-daf47a5963c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:34.367471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53761838-bc32-46ff-8623-daf47a5963c0 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:34.377218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 919/1837] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:34.401311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379d6f7-9485-47e8-9de5-cf9c553111a7 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:34.466990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379d6f7-9485-47e8-9de5-cf9c553111a7 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:34.472225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:34.472913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:34.473286 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:34.473691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] PATH_INFO: `/v3/users/2f60a2965fe4432592f392f986857bee` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:34.478233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f60a2965fe4432592f392f986857bee)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:34.480305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:34.528087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f60a2965fe4432592f392f986857bee'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:34.528731 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f60a2965fe4432592f392f986857bee'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:34.535625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f60a2965fe4432592f392f986857bee'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:34.547331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f60a2965fe4432592f392f986857bee'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:34.554842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f60a2965fe4432592f392f986857bee'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:34.560885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 919/1838] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:34 2022] DELETE /identity/v3/users/2f60a2965fe4432592f392f986857bee => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:34.565317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143c353-8217-4e7c-ae26-cbe7252c5f25 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:34.566270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143c353-8217-4e7c-ae26-cbe7252c5f25 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:34.566402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143c353-8217-4e7c-ae26-cbe7252c5f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:34.590274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-3143c353-8217-4e7c-ae26-cbe7252c5f25 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:34.591504 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3143c353-8217-4e7c-ae26-cbe7252c5f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:34.663662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 920/1839] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:34.673319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:34.675413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:34.716455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:34.735175 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982b37c-9e85-4948-aaa0-146c716db3a9 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:34.739353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21228495-8984-4b2d-81e8-cae1b051a817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:34.739964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21228495-8984-4b2d-81e8-cae1b051a817 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:34.740248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21228495-8984-4b2d-81e8-cae1b051a817 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:34.740526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21228495-8984-4b2d-81e8-cae1b051a817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:34.749459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21228495-8984-4b2d-81e8-cae1b051a817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:34.751052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21228495-8984-4b2d-81e8-cae1b051a817 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:34.761263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 920/1840] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:35.482512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143c353-8217-4e7c-ae26-cbe7252c5f25 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:35.484430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3143c353-8217-4e7c-ae26-cbe7252c5f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:35.500562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143c353-8217-4e7c-ae26-cbe7252c5f25 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:35.506461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:35.507198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:35.507576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:35.507958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] PATH_INFO: `/v3/projects/f9a7e11e3b334c7faf028d8a297f726e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:35.512549 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a7e11e3b334c7faf028d8a297f726e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:35.515663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:35.594335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a7e11e3b334c7faf028d8a297f726e', 'request_id': 'req-142d8069-9445-42c3-8bbf-fbfb552efb5d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:35.624835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a7e11e3b334c7faf028d8a297f726e', 'request_id': 'req-142d8069-9445-42c3-8bbf-fbfb552efb5d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:35.633439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a7e11e3b334c7faf028d8a297f726e', 'request_id': 'req-142d8069-9445-42c3-8bbf-fbfb552efb5d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:35.651886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] The token cache is being invalidate because project f9a7e11e3b334c7faf028d8a297f726e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:35.652272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:35.653023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-142d8069-9445-42c3-8bbf-fbfb552efb5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:35.660990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 921/1841] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:35 2022] DELETE /identity/v3/projects/f9a7e11e3b334c7faf028d8a297f726e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:35.665799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680a39-e9e6-4253-b708-6b3fddb85c47 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:35.666087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680a39-e9e6-4253-b708-6b3fddb85c47 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:35.666363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680a39-e9e6-4253-b708-6b3fddb85c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:35.691471 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-14680a39-e9e6-4253-b708-6b3fddb85c47 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:35.705590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-14680a39-e9e6-4253-b708-6b3fddb85c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:35.776683 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 921/1842] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:35.785123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e3639-f7ef-4c48-9b7b-fbb762dd3788 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:35.785478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e3639-f7ef-4c48-9b7b-fbb762dd3788 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:35.785861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e3639-f7ef-4c48-9b7b-fbb762dd3788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:35.807251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-991e3639-f7ef-4c48-9b7b-fbb762dd3788 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:35.808234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-991e3639-f7ef-4c48-9b7b-fbb762dd3788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:35.823186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 922/1843] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:35.824237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14680a39-e9e6-4253-b708-6b3fddb85c47 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:35.835420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14680a39-e9e6-4253-b708-6b3fddb85c47 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:35.839970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08d6a4-2b80-45f2-a79f-573fa818b231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:35.840570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08d6a4-2b80-45f2-a79f-573fa818b231 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:35.840849 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08d6a4-2b80-45f2-a79f-573fa818b231 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:35.841118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08d6a4-2b80-45f2-a79f-573fa818b231 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:35.841845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08d6a4-2b80-45f2-a79f-573fa818b231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:52:35.842146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08d6a4-2b80-45f2-a79f-573fa818b231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:35.843998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08d6a4-2b80-45f2-a79f-573fa818b231 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:35.856827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 922/1844] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:52:35.861194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e3639-f7ef-4c48-9b7b-fbb762dd3788 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:35.871469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e3639-f7ef-4c48-9b7b-fbb762dd3788 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:35.879673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb935e-fab5-414b-80b9-c3281486cda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:35.879938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb935e-fab5-414b-80b9-c3281486cda6 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:35.880246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb935e-fab5-414b-80b9-c3281486cda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:35.880466 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb935e-fab5-414b-80b9-c3281486cda6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:35.881100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb935e-fab5-414b-80b9-c3281486cda6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:35.883032 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb935e-fab5-414b-80b9-c3281486cda6 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:35.913408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 923/1845] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:35 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:52:35.917881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08d6a4-2b80-45f2-a79f-573fa818b231 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:35.928385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08d6a4-2b80-45f2-a79f-573fa818b231 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:35.993241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f8fd-9516-4851-a980-283dbe2000c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:35.993897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f8fd-9516-4851-a980-283dbe2000c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:35.994172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f8fd-9516-4851-a980-283dbe2000c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:35.994441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f8fd-9516-4851-a980-283dbe2000c4 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:35.995083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292f8fd-9516-4851-a980-283dbe2000c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:35.996993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292f8fd-9516-4851-a980-283dbe2000c4 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:36.020859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 923/1846] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:35 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:52:36.024921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb935e-fab5-414b-80b9-c3281486cda6 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:36.038406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb935e-fab5-414b-80b9-c3281486cda6 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:36.043135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263648c-9cb4-41d0-b468-b94b1b6217ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:36.043744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263648c-9cb4-41d0-b468-b94b1b6217ef admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:36.044021 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263648c-9cb4-41d0-b468-b94b1b6217ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:36.044297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263648c-9cb4-41d0-b468-b94b1b6217ef admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:36.044938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263648c-9cb4-41d0-b468-b94b1b6217ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:36.046775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263648c-9cb4-41d0-b468-b94b1b6217ef admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:36.049257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:36.065167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 924/1847] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:36.067045 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f8fd-9516-4851-a980-283dbe2000c4 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:36.077311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f8fd-9516-4851-a980-283dbe2000c4 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:36.082614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e77b35-9c20-4cd8-b0ad-9a4d1558a159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:36.083223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e77b35-9c20-4cd8-b0ad-9a4d1558a159 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:36.083506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e77b35-9c20-4cd8-b0ad-9a4d1558a159 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:36.083780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e77b35-9c20-4cd8-b0ad-9a4d1558a159 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:36.084436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e77b35-9c20-4cd8-b0ad-9a4d1558a159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:36.086082 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e77b35-9c20-4cd8-b0ad-9a4d1558a159 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:36.098038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 924/1848] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:36.102517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263648c-9cb4-41d0-b468-b94b1b6217ef admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:36.116498 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263648c-9cb4-41d0-b468-b94b1b6217ef admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:36.121674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1962830-1cd3-4587-bd81-85583863e3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:36.122409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1962830-1cd3-4587-bd81-85583863e3ac admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:36.122758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1962830-1cd3-4587-bd81-85583863e3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:36.123102 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1962830-1cd3-4587-bd81-85583863e3ac admin admin] PATH_INFO: `/v3/projects/b617943442e745b0807ccc38c814fd04/users/095f83722b8a4959ac0c5c0d527a477f/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:36.139742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1962830-1cd3-4587-bd81-85583863e3ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b617943442e745b0807ccc38c814fd04, user_id=095f83722b8a4959ac0c5c0d527a477f, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:36.142581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1962830-1cd3-4587-bd81-85583863e3ac admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:36.164000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 925/1849] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:36 2022] PUT /identity/v3/projects/b617943442e745b0807ccc38c814fd04/users/095f83722b8a4959ac0c5c0d527a477f/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:36.176571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd721f5-448d-4b86-8340-290a9d5028fd None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:36.176571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd721f5-448d-4b86-8340-290a9d5028fd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:36.176571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd721f5-448d-4b86-8340-290a9d5028fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:36.208233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-dfd721f5-448d-4b86-8340-290a9d5028fd None None] MFA Rules not processed for user `095f83722b8a4959ac0c5c0d527a477f`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:36.209581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-dfd721f5-448d-4b86-8340-290a9d5028fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:36.245824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:36.279395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 925/1850] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:36.290138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1962830-1cd3-4587-bd81-85583863e3ac admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:36.291803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d1962830-1cd3-4587-bd81-85583863e3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:36.350272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1962830-1cd3-4587-bd81-85583863e3ac admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:36.353842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c52da-8a2f-4361-9d62-52bb2d77c89e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:36.354612 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c52da-8a2f-4361-9d62-52bb2d77c89e service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:36.354722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c52da-8a2f-4361-9d62-52bb2d77c89e service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:36.354988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c52da-8a2f-4361-9d62-52bb2d77c89e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:36.364233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c52da-8a2f-4361-9d62-52bb2d77c89e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:36.365829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c52da-8a2f-4361-9d62-52bb2d77c89e service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:36.374759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 926/1851] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:41.078651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd721f5-448d-4b86-8340-290a9d5028fd None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:41.080058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-dfd721f5-448d-4b86-8340-290a9d5028fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:41.150454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd721f5-448d-4b86-8340-290a9d5028fd None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:41.155945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:41.156746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:41.157305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:41.157663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] PATH_INFO: `/v3/users/939dccdf404741eaacb60c298f0d9c7f` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:41.160019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=939dccdf404741eaacb60c298f0d9c7f)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:41.163252 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:41.229413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939dccdf404741eaacb60c298f0d9c7f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:41.230193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939dccdf404741eaacb60c298f0d9c7f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:41.241581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939dccdf404741eaacb60c298f0d9c7f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:41.264568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939dccdf404741eaacb60c298f0d9c7f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:41.284383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939dccdf404741eaacb60c298f0d9c7f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:41.297278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 926/1852] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:41 2022] DELETE /identity/v3/users/939dccdf404741eaacb60c298f0d9c7f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:41.303277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c091b-ac23-47d1-95d0-51dd5ca4372f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:41.303696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c091b-ac23-47d1-95d0-51dd5ca4372f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:41.304186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c091b-ac23-47d1-95d0-51dd5ca4372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:41.326453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-be9c091b-ac23-47d1-95d0-51dd5ca4372f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:41.328074 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-be9c091b-ac23-47d1-95d0-51dd5ca4372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:41.419732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 927/1853] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:41.432154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:41.442753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:41.527954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8598b6-b129-4111-a8e2-5de7e70c69fc admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:41.531833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea51a05-b7d2-4914-95b8-7686f8c31b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:41.532441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea51a05-b7d2-4914-95b8-7686f8c31b60 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:41.532716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea51a05-b7d2-4914-95b8-7686f8c31b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:41.532984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea51a05-b7d2-4914-95b8-7686f8c31b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:41.540360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:41.543063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea51a05-b7d2-4914-95b8-7686f8c31b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:41.545056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea51a05-b7d2-4914-95b8-7686f8c31b60 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:41.566422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 927/1854] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:41.924251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c091b-ac23-47d1-95d0-51dd5ca4372f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:41.927107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-be9c091b-ac23-47d1-95d0-51dd5ca4372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:41.962025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c091b-ac23-47d1-95d0-51dd5ca4372f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:41.972458 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:41.975367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:41.975658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:41.975945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] PATH_INFO: `/v3/projects/e5cb00920a7e4291b6ee8dd70e4d6026` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:41.977130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5cb00920a7e4291b6ee8dd70e4d6026)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:41.979107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:42.037539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5cb00920a7e4291b6ee8dd70e4d6026', 'request_id': 'req-7381dc48-8a90-4b56-a1bb-a137250b74ea'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:42.045376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5cb00920a7e4291b6ee8dd70e4d6026', 'request_id': 'req-7381dc48-8a90-4b56-a1bb-a137250b74ea'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:42.052658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5cb00920a7e4291b6ee8dd70e4d6026', 'request_id': 'req-7381dc48-8a90-4b56-a1bb-a137250b74ea'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:42.059862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:42.063893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] The token cache is being invalidate because project e5cb00920a7e4291b6ee8dd70e4d6026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:42.064194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:42.065856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:42.067474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 928/1855] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:41 2022] DELETE /identity/v3/projects/e5cb00920a7e4291b6ee8dd70e4d6026 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:42.071782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea51a05-b7d2-4914-95b8-7686f8c31b60 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:42.081183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea51a05-b7d2-4914-95b8-7686f8c31b60 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:42.148909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:42.150066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:42.150579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:42.151318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] PATH_INFO: `/v3/users/f5515e02510846fe955703282e6cdc3e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:42.152824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5515e02510846fe955703282e6cdc3e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:42.155080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:42.206500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5515e02510846fe955703282e6cdc3e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:42.207280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5515e02510846fe955703282e6cdc3e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:42.224395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5515e02510846fe955703282e6cdc3e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:42.240298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5515e02510846fe955703282e6cdc3e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:42.252575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1a82a9d2-d276-4216-b169-728c30edd6a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5515e02510846fe955703282e6cdc3e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:42.263692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 928/1856] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:42 2022] DELETE /identity/v3/users/f5515e02510846fe955703282e6cdc3e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:43.373112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:43.374395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:43.388464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381dc48-8a90-4b56-a1bb-a137250b74ea admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:43.466916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:43.467552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:43.467837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:43.468369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] PATH_INFO: `/v3/projects/f9d0841fd6a74571af7ebd65101ea53a` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:43.469546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d0841fd6a74571af7ebd65101ea53a)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:43.472008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:43.518928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d0841fd6a74571af7ebd65101ea53a', 'request_id': 'req-27811bb4-d86d-4afc-a222-5cc717a240d1'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:43.526975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d0841fd6a74571af7ebd65101ea53a', 'request_id': 'req-27811bb4-d86d-4afc-a222-5cc717a240d1'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:43.534229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d0841fd6a74571af7ebd65101ea53a', 'request_id': 'req-27811bb4-d86d-4afc-a222-5cc717a240d1'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:43.547344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] The token cache is being invalidate because project f9d0841fd6a74571af7ebd65101ea53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:43.547673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:43.548575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-27811bb4-d86d-4afc-a222-5cc717a240d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:43.554575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 929/1857] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:43 2022] DELETE /identity/v3/projects/f9d0841fd6a74571af7ebd65101ea53a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:43.556192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd473d-071d-40de-9dd8-358aff66a515 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:43.556728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd473d-071d-40de-9dd8-358aff66a515 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:43.556817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd473d-071d-40de-9dd8-358aff66a515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:43.582916 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-3fdd473d-071d-40de-9dd8-358aff66a515 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:43.584139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3fdd473d-071d-40de-9dd8-358aff66a515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:43.648527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 929/1858] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:43.657642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae151-71dd-4d07-9e8f-b2ab9a377dc9 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:43.657926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae151-71dd-4d07-9e8f-b2ab9a377dc9 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:43.658201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae151-71dd-4d07-9e8f-b2ab9a377dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:43.679449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ed5ae151-71dd-4d07-9e8f-b2ab9a377dc9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:43.680732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ed5ae151-71dd-4d07-9e8f-b2ab9a377dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:43.688956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 930/1859] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:43.693377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd473d-071d-40de-9dd8-358aff66a515 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:43.703852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd473d-071d-40de-9dd8-358aff66a515 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:43.709083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc0614-ace2-4d57-9d10-4bc0f82c8aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:43.709745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc0614-ace2-4d57-9d10-4bc0f82c8aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:43.710056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc0614-ace2-4d57-9d10-4bc0f82c8aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:43.710328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc0614-ace2-4d57-9d10-4bc0f82c8aa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:43.710990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc0614-ace2-4d57-9d10-4bc0f82c8aa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:52:43.711298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc0614-ace2-4d57-9d10-4bc0f82c8aa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:43.712975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc0614-ace2-4d57-9d10-4bc0f82c8aa4 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:43.726564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 930/1860] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:52:43.730849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ae151-71dd-4d07-9e8f-b2ab9a377dc9 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:43.740714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ae151-71dd-4d07-9e8f-b2ab9a377dc9 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:43.745301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76526573-594e-48f1-beda-5ea131cd689b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:43.745944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76526573-594e-48f1-beda-5ea131cd689b admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:43.746218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76526573-594e-48f1-beda-5ea131cd689b admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:43.746481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76526573-594e-48f1-beda-5ea131cd689b admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:43.747115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76526573-594e-48f1-beda-5ea131cd689b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:43.749036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76526573-594e-48f1-beda-5ea131cd689b admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:43.780703 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 931/1861] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:43 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:43.782182 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc0614-ace2-4d57-9d10-4bc0f82c8aa4 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:43.790328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:43.796294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc0614-ace2-4d57-9d10-4bc0f82c8aa4 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:43.869926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d3417-4262-433d-9b8d-a71894a6af0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:43.870551 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d3417-4262-433d-9b8d-a71894a6af0d admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:43.870828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d3417-4262-433d-9b8d-a71894a6af0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:43.871095 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d3417-4262-433d-9b8d-a71894a6af0d admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:43.871788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d3417-4262-433d-9b8d-a71894a6af0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:43.885634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d3417-4262-433d-9b8d-a71894a6af0d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:43.943467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 931/1862] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:43 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:43.947752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76526573-594e-48f1-beda-5ea131cd689b admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:43.961285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76526573-594e-48f1-beda-5ea131cd689b admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:43.966198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883ba77-814c-49cf-aecd-5ca11f1bcd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:43.966932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883ba77-814c-49cf-aecd-5ca11f1bcd6a admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:43.967216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883ba77-814c-49cf-aecd-5ca11f1bcd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:43.967713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883ba77-814c-49cf-aecd-5ca11f1bcd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:43.968909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883ba77-814c-49cf-aecd-5ca11f1bcd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:43.971132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883ba77-814c-49cf-aecd-5ca11f1bcd6a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:43.995188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 932/1863] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:43.999813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d3417-4262-433d-9b8d-a71894a6af0d admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:44.015503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d3417-4262-433d-9b8d-a71894a6af0d admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:44.023363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7c8e8-ccb1-4061-89ae-011549e08af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:44.024221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7c8e8-ccb1-4061-89ae-011549e08af1 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:44.024669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7c8e8-ccb1-4061-89ae-011549e08af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:44.025094 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7c8e8-ccb1-4061-89ae-011549e08af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:44.025944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7c8e8-ccb1-4061-89ae-011549e08af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:44.027690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7c8e8-ccb1-4061-89ae-011549e08af1 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:44.043261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 932/1864] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:44.047334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883ba77-814c-49cf-aecd-5ca11f1bcd6a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:44.060912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883ba77-814c-49cf-aecd-5ca11f1bcd6a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:44.067475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c608a-637b-42ae-9c81-315c4e4f7bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:44.068699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c608a-637b-42ae-9c81-315c4e4f7bff admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:44.068971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c608a-637b-42ae-9c81-315c4e4f7bff admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:44.069344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c608a-637b-42ae-9c81-315c4e4f7bff admin admin] PATH_INFO: `/v3/projects/43e25ec562ca473ea6a82d44964fb243/users/e822a6ee45b94fc089079d648eeca59f/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:44.087601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c608a-637b-42ae-9c81-315c4e4f7bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e25ec562ca473ea6a82d44964fb243, user_id=e822a6ee45b94fc089079d648eeca59f, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:44.091081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c608a-637b-42ae-9c81-315c4e4f7bff admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:44.108647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 933/1865] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:44 2022] PUT /identity/v3/projects/43e25ec562ca473ea6a82d44964fb243/users/e822a6ee45b94fc089079d648eeca59f/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:44.119159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2d89f-4df0-4810-97de-8221b8c378dc None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:44.119446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2d89f-4df0-4810-97de-8221b8c378dc None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:44.119781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2d89f-4df0-4810-97de-8221b8c378dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:44.157139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-dad2d89f-4df0-4810-97de-8221b8c378dc None None] MFA Rules not processed for user `e822a6ee45b94fc089079d648eeca59f`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:44.158376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-dad2d89f-4df0-4810-97de-8221b8c378dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:44.230926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 933/1866] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:44.235971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c608a-637b-42ae-9c81-315c4e4f7bff admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:44.265456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:44.301163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c608a-637b-42ae-9c81-315c4e4f7bff admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:44.305875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8246c6d-c7f8-4753-b5ab-9ad1e17537a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:44.306474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246c6d-c7f8-4753-b5ab-9ad1e17537a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:44.306754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246c6d-c7f8-4753-b5ab-9ad1e17537a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:44.307024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246c6d-c7f8-4753-b5ab-9ad1e17537a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:44.307655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8246c6d-c7f8-4753-b5ab-9ad1e17537a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:44.309589 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8246c6d-c7f8-4753-b5ab-9ad1e17537a0 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:44.336535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 934/1867] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:44 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:44.340744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2d89f-4df0-4810-97de-8221b8c378dc None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:44.351549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2d89f-4df0-4810-97de-8221b8c378dc None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:44.408400 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2289a5b-2eb6-4bf1-b6f7-e30ac884195b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:44.409010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2289a5b-2eb6-4bf1-b6f7-e30ac884195b admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:44.409287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2289a5b-2eb6-4bf1-b6f7-e30ac884195b admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:44.409595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2289a5b-2eb6-4bf1-b6f7-e30ac884195b admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:44.410245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2289a5b-2eb6-4bf1-b6f7-e30ac884195b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:44.412178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2289a5b-2eb6-4bf1-b6f7-e30ac884195b admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:44.433099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:44.442766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 934/1868] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:44 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:44.443357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8246c6d-c7f8-4753-b5ab-9ad1e17537a0 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:44.454223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8246c6d-c7f8-4753-b5ab-9ad1e17537a0 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:44.460028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c08044-8037-42c4-b1ae-c87222dd68a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:44.460641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c08044-8037-42c4-b1ae-c87222dd68a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:44.460917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c08044-8037-42c4-b1ae-c87222dd68a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:44.461185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c08044-8037-42c4-b1ae-c87222dd68a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:44.461957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c08044-8037-42c4-b1ae-c87222dd68a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:44.463570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c08044-8037-42c4-b1ae-c87222dd68a6 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:44.480480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 935/1869] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:44.484365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2289a5b-2eb6-4bf1-b6f7-e30ac884195b admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:44.496017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2289a5b-2eb6-4bf1-b6f7-e30ac884195b admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:44.502912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12617cf7-fd95-4190-b36e-2015c6ab8985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:44.503863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12617cf7-fd95-4190-b36e-2015c6ab8985 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:44.504325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12617cf7-fd95-4190-b36e-2015c6ab8985 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:44.504826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12617cf7-fd95-4190-b36e-2015c6ab8985 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:44.505738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12617cf7-fd95-4190-b36e-2015c6ab8985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:44.507804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12617cf7-fd95-4190-b36e-2015c6ab8985 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:44.519528 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 935/1870] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:44.523643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c08044-8037-42c4-b1ae-c87222dd68a6 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:44.534649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c08044-8037-42c4-b1ae-c87222dd68a6 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:44.539455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ad623-2d6a-40f4-ab82-047266507997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:44.540254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ad623-2d6a-40f4-ab82-047266507997 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:44.540605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ad623-2d6a-40f4-ab82-047266507997 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:44.541054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ad623-2d6a-40f4-ab82-047266507997 admin admin] PATH_INFO: `/v3/projects/672f2451cf0d4219b42d6bd3ffa6f473/users/eaed5d9f8f584a69935ae89e8c6432c2/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:44.570163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ad623-2d6a-40f4-ab82-047266507997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=672f2451cf0d4219b42d6bd3ffa6f473, user_id=eaed5d9f8f584a69935ae89e8c6432c2, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:44.572896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ad623-2d6a-40f4-ab82-047266507997 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:44.586113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 936/1871] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:44 2022] PUT /identity/v3/projects/672f2451cf0d4219b42d6bd3ffa6f473/users/eaed5d9f8f584a69935ae89e8c6432c2/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:44.597714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59fbcb-73dd-48fe-af0f-ed012415abd1 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:44.598158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59fbcb-73dd-48fe-af0f-ed012415abd1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:44.598327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59fbcb-73dd-48fe-af0f-ed012415abd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:44.628842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2d59fbcb-73dd-48fe-af0f-ed012415abd1 None None] MFA Rules not processed for user `eaed5d9f8f584a69935ae89e8c6432c2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:44.630105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2d59fbcb-73dd-48fe-af0f-ed012415abd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:44.704794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 936/1872] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:44.710108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ad623-2d6a-40f4-ab82-047266507997 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:44.835129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ad623-2d6a-40f4-ab82-047266507997 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:44.864518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2644430-1d04-4cad-b273-88fc1f27aa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:44.865480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2644430-1d04-4cad-b273-88fc1f27aa5f admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:44.866251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2644430-1d04-4cad-b273-88fc1f27aa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:44.866752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2644430-1d04-4cad-b273-88fc1f27aa5f admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:44.867686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2644430-1d04-4cad-b273-88fc1f27aa5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:44.870325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2644430-1d04-4cad-b273-88fc1f27aa5f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:44.905544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 937/1873] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:44 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:44.909550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59fbcb-73dd-48fe-af0f-ed012415abd1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:44.923675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59fbcb-73dd-48fe-af0f-ed012415abd1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:45.059855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d59e3-6e4b-4ed5-a5b0-7610a227524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:45.060489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d59e3-6e4b-4ed5-a5b0-7610a227524c admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:45.061047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d59e3-6e4b-4ed5-a5b0-7610a227524c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:45.061047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d59e3-6e4b-4ed5-a5b0-7610a227524c admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:45.061832 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d59e3-6e4b-4ed5-a5b0-7610a227524c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:45.063778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d59e3-6e4b-4ed5-a5b0-7610a227524c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:45.090505 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 937/1874] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:44 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:45.094683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2644430-1d04-4cad-b273-88fc1f27aa5f admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:45.114203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2644430-1d04-4cad-b273-88fc1f27aa5f admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:45.140067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928efdd4-0f75-4dd7-9998-1e0c147babf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:45.140698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928efdd4-0f75-4dd7-9998-1e0c147babf5 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:45.140977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928efdd4-0f75-4dd7-9998-1e0c147babf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:45.141248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928efdd4-0f75-4dd7-9998-1e0c147babf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:45.141930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928efdd4-0f75-4dd7-9998-1e0c147babf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:45.145274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928efdd4-0f75-4dd7-9998-1e0c147babf5 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:45.167777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 938/1875] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:45.174276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d59e3-6e4b-4ed5-a5b0-7610a227524c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:45.187332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d59e3-6e4b-4ed5-a5b0-7610a227524c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:45.221646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4da79-445a-4300-864c-d7d9f142a1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:45.221646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4da79-445a-4300-864c-d7d9f142a1f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:45.221646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4da79-445a-4300-864c-d7d9f142a1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:45.221646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4da79-445a-4300-864c-d7d9f142a1f9 admin admin] PATH_INFO: `/v3/projects/7853ff311f20431399e382138f6e391d/users/244c2b2f81e444e98f27eef0dea0e6d0/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:45.253717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4da79-445a-4300-864c-d7d9f142a1f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7853ff311f20431399e382138f6e391d, user_id=244c2b2f81e444e98f27eef0dea0e6d0, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:45.256564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4da79-445a-4300-864c-d7d9f142a1f9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:45.278240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 938/1876] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:45 2022] PUT /identity/v3/projects/7853ff311f20431399e382138f6e391d/users/244c2b2f81e444e98f27eef0dea0e6d0/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:45.282186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928efdd4-0f75-4dd7-9998-1e0c147babf5 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:45.297645 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928efdd4-0f75-4dd7-9998-1e0c147babf5 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:45.398053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:45.410706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af68a23-f018-4bbd-b21f-564ed71844a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:45.411771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af68a23-f018-4bbd-b21f-564ed71844a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:45.412173 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af68a23-f018-4bbd-b21f-564ed71844a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:45.412519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af68a23-f018-4bbd-b21f-564ed71844a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:45.413243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af68a23-f018-4bbd-b21f-564ed71844a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:45.414955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af68a23-f018-4bbd-b21f-564ed71844a6 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:45.426477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 939/1877] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:45.432114 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4da79-445a-4300-864c-d7d9f142a1f9 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:45.439817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18beae-3608-47f3-bac4-e976c5505d44 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:45.441577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18beae-3608-47f3-bac4-e976c5505d44 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:45.443108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18beae-3608-47f3-bac4-e976c5505d44 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:45.447246 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4da79-445a-4300-864c-d7d9f142a1f9 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:45.449473 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 940/1878] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:52:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:52:45.454356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b1e1e-9e70-47ab-a92e-f883a4df1c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:45.455296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b1e1e-9e70-47ab-a92e-f883a4df1c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:45.456328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b1e1e-9e70-47ab-a92e-f883a4df1c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:45.456830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b1e1e-9e70-47ab-a92e-f883a4df1c61 admin admin] PATH_INFO: `/v3/projects/7853ff311f20431399e382138f6e391d/users/244c2b2f81e444e98f27eef0dea0e6d0/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:45.459051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b1e1e-9e70-47ab-a92e-f883a4df1c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7853ff311f20431399e382138f6e391d, user_id=244c2b2f81e444e98f27eef0dea0e6d0, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:45.461972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b1e1e-9e70-47ab-a92e-f883a4df1c61 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:45.475875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 939/1879] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:45 2022] PUT /identity/v3/projects/7853ff311f20431399e382138f6e391d/users/244c2b2f81e444e98f27eef0dea0e6d0/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:45.486541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9c159-abab-4690-9081-3f91ced20714 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:45.486711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9c159-abab-4690-9081-3f91ced20714 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:45.487086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9c159-abab-4690-9081-3f91ced20714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:45.504410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b7cd3-fd72-4a76-8f3a-92554ac96872 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:45.504702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b7cd3-fd72-4a76-8f3a-92554ac96872 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:45.504971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b7cd3-fd72-4a76-8f3a-92554ac96872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:45.534161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-9ad9c159-abab-4690-9081-3f91ced20714 None None] MFA Rules not processed for user `244c2b2f81e444e98f27eef0dea0e6d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:45.535945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9ad9c159-abab-4690-9081-3f91ced20714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:45.558180 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b66b7cd3-fd72-4a76-8f3a-92554ac96872 None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:45.559474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b66b7cd3-fd72-4a76-8f3a-92554ac96872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:45.608920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:45.725741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 940/1880] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:52:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:45.736571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 941/1881] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:45.749576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b7cd3-fd72-4a76-8f3a-92554ac96872 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:45.753304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b66b7cd3-fd72-4a76-8f3a-92554ac96872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:45.915277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b7cd3-fd72-4a76-8f3a-92554ac96872 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:45.923333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453e13d-b2b4-458a-9169-f219aec5d9f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:45.924625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453e13d-b2b4-458a-9169-f219aec5d9f1 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:45.925287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453e13d-b2b4-458a-9169-f219aec5d9f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:45.925436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453e13d-b2b4-458a-9169-f219aec5d9f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:45.946608 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453e13d-b2b4-458a-9169-f219aec5d9f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:45.948117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453e13d-b2b4-458a-9169-f219aec5d9f1 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:45.981260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 941/1882] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:46.087597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9c159-abab-4690-9081-3f91ced20714 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:46.092335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9ad9c159-abab-4690-9081-3f91ced20714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:46.109512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9c159-abab-4690-9081-3f91ced20714 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:46.116339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80b6d4-f120-4c50-9ab3-c6289b4391d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:46.116925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80b6d4-f120-4c50-9ab3-c6289b4391d6 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:46.117777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80b6d4-f120-4c50-9ab3-c6289b4391d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:46.118060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80b6d4-f120-4c50-9ab3-c6289b4391d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:46.197641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80b6d4-f120-4c50-9ab3-c6289b4391d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:46.199105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80b6d4-f120-4c50-9ab3-c6289b4391d6 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:46.216636 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 942/1883] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:46.382774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453e13d-b2b4-458a-9169-f219aec5d9f1 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:46.384247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9453e13d-b2b4-458a-9169-f219aec5d9f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:46.412184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453e13d-b2b4-458a-9169-f219aec5d9f1 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:46.429356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2fdf7-7df3-4b35-9d6f-ed1f360daeee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:46.430042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2fdf7-7df3-4b35-9d6f-ed1f360daeee service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:46.430380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2fdf7-7df3-4b35-9d6f-ed1f360daeee service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:46.430582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2fdf7-7df3-4b35-9d6f-ed1f360daeee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:46.453870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2fdf7-7df3-4b35-9d6f-ed1f360daeee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:46.455621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2fdf7-7df3-4b35-9d6f-ed1f360daeee service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:46.483035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 942/1884] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:52:53.699017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80b6d4-f120-4c50-9ab3-c6289b4391d6 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:53.762387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80b6d4-f120-4c50-9ab3-c6289b4391d6 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:53.769624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:53.770297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:53.770519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:53.770785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] PATH_INFO: `/v3/users/e822a6ee45b94fc089079d648eeca59f` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:53.771888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e822a6ee45b94fc089079d648eeca59f)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:53.774088 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:53.786541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:53.819096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e822a6ee45b94fc089079d648eeca59f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:53.819743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e822a6ee45b94fc089079d648eeca59f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:53.826510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e822a6ee45b94fc089079d648eeca59f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:53.834164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e822a6ee45b94fc089079d648eeca59f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:53.834773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e822a6ee45b94fc089079d648eeca59f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:53.851827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 943/1885] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:53 2022] DELETE /identity/v3/users/e822a6ee45b94fc089079d648eeca59f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:53.856482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d0b4e-be8d-4497-b5c8-4d8fc42d48da None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:53.856979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d0b4e-be8d-4497-b5c8-4d8fc42d48da None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:53.857075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d0b4e-be8d-4497-b5c8-4d8fc42d48da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:53.875447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-237d0b4e-be8d-4497-b5c8-4d8fc42d48da None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:53.876621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-237d0b4e-be8d-4497-b5c8-4d8fc42d48da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:53.934118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 943/1886] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:53.944298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:53.945694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:53.971456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d0b4e-be8d-4497-b5c8-4d8fc42d48da None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:53.972837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-237d0b4e-be8d-4497-b5c8-4d8fc42d48da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:53.984566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d0b4e-be8d-4497-b5c8-4d8fc42d48da None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:53.989115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:53.989763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:53.990046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:53.990318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] PATH_INFO: `/v3/users/095f83722b8a4959ac0c5c0d527a477f` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:53.991438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=095f83722b8a4959ac0c5c0d527a477f)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:53.993373 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:54.005221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf98f7-cc48-435b-83c7-a14f4a0e346f admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:54.008784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd8c60-bf1d-4a7a-8197-6a17d2546a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:54.009382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd8c60-bf1d-4a7a-8197-6a17d2546a3b service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:54.009698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd8c60-bf1d-4a7a-8197-6a17d2546a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:54.009965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd8c60-bf1d-4a7a-8197-6a17d2546a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:54.020412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd8c60-bf1d-4a7a-8197-6a17d2546a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:54.021971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd8c60-bf1d-4a7a-8197-6a17d2546a3b service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:54.032424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 944/1887] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:54.041799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095f83722b8a4959ac0c5c0d527a477f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.042483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095f83722b8a4959ac0c5c0d527a477f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.043702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:54.051147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095f83722b8a4959ac0c5c0d527a477f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.063416 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095f83722b8a4959ac0c5c0d527a477f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.071652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095f83722b8a4959ac0c5c0d527a477f'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.077945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 944/1888] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:53 2022] DELETE /identity/v3/users/095f83722b8a4959ac0c5c0d527a477f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:54.083207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d9a7-8259-4e56-a4cc-86af29dbe2e1 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:54.083486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d9a7-8259-4e56-a4cc-86af29dbe2e1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:54.083756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d9a7-8259-4e56-a4cc-86af29dbe2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:54.104825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-fae8d9a7-8259-4e56-a4cc-86af29dbe2e1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:54.106264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-fae8d9a7-8259-4e56-a4cc-86af29dbe2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:54.180260 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 945/1889] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:54.201938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:54.203693 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:54.291108 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa6508-9262-4ec1-9063-a39e2f8109b7 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:54.294751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895c87-d1a5-45f2-bf23-323e125f88ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:54.295377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895c87-d1a5-45f2-bf23-323e125f88ff service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:54.295808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895c87-d1a5-45f2-bf23-323e125f88ff service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:54.296094 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895c87-d1a5-45f2-bf23-323e125f88ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:54.306334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee895c87-d1a5-45f2-bf23-323e125f88ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:54.307782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee895c87-d1a5-45f2-bf23-323e125f88ff service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:54.318327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 945/1890] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:54.462234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8d9a7-8259-4e56-a4cc-86af29dbe2e1 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:54.463737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-fae8d9a7-8259-4e56-a4cc-86af29dbe2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:54.495864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8d9a7-8259-4e56-a4cc-86af29dbe2e1 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:54.520919 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:54.521801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:54.522287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:54.523487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] PATH_INFO: `/v3/projects/43e25ec562ca473ea6a82d44964fb243` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:54.527752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43e25ec562ca473ea6a82d44964fb243)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:54.529921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:54.589978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:54.613073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43e25ec562ca473ea6a82d44964fb243', 'request_id': 'req-b0dd56a3-0854-45e1-8aea-9f45536b53c6'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.645436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43e25ec562ca473ea6a82d44964fb243', 'request_id': 'req-b0dd56a3-0854-45e1-8aea-9f45536b53c6'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.655910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43e25ec562ca473ea6a82d44964fb243', 'request_id': 'req-b0dd56a3-0854-45e1-8aea-9f45536b53c6'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.701128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] The token cache is being invalidate because project 43e25ec562ca473ea6a82d44964fb243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:54.702479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.703344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.707065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 946/1891] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:54 2022] DELETE /identity/v3/projects/43e25ec562ca473ea6a82d44964fb243 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:54.711228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895c87-d1a5-45f2-bf23-323e125f88ff service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:54.723991 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895c87-d1a5-45f2-bf23-323e125f88ff service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:54.822916 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:54.824115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:54.824382 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:54.824711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] PATH_INFO: `/v3/users/eaed5d9f8f584a69935ae89e8c6432c2` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:54.826134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaed5d9f8f584a69935ae89e8c6432c2)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:54.829625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:54.874383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:54.877248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:54.901443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd56a3-0854-45e1-8aea-9f45536b53c6 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:54.915546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:54.916204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:54.916544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:54.916823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] PATH_INFO: `/v3/projects/b617943442e745b0807ccc38c814fd04` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:54.918479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b617943442e745b0807ccc38c814fd04)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:54.920530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:54.937049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaed5d9f8f584a69935ae89e8c6432c2'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.938251 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaed5d9f8f584a69935ae89e8c6432c2'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.946198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaed5d9f8f584a69935ae89e8c6432c2'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.957330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:54.959842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaed5d9f8f584a69935ae89e8c6432c2'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.976501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5e1246c9-5d72-4d04-a009-820a01daa3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaed5d9f8f584a69935ae89e8c6432c2'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:54.987812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 946/1892] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:54 2022] DELETE /identity/v3/users/eaed5d9f8f584a69935ae89e8c6432c2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:54.995698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b617943442e745b0807ccc38c814fd04', 'request_id': 'req-c4544f50-92fe-477a-a09d-a524c0a574fc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.006969 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b617943442e745b0807ccc38c814fd04', 'request_id': 'req-c4544f50-92fe-477a-a09d-a524c0a574fc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.023208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b617943442e745b0807ccc38c814fd04', 'request_id': 'req-c4544f50-92fe-477a-a09d-a524c0a574fc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.052034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] The token cache is being invalidate because project b617943442e745b0807ccc38c814fd04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:55.052397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.053152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.056012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 947/1893] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:54 2022] DELETE /identity/v3/projects/b617943442e745b0807ccc38c814fd04 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:55.061860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68246b8d-c098-4d0c-8b48-1003c241675a None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.062287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68246b8d-c098-4d0c-8b48-1003c241675a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.062863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68246b8d-c098-4d0c-8b48-1003c241675a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.103491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-68246b8d-c098-4d0c-8b48-1003c241675a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:55.108316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-68246b8d-c098-4d0c-8b48-1003c241675a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:55.174051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:55.175621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:55.237328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 947/1894] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:55.245703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac47ad3-1e72-4e81-a1ce-2194befca18c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.246156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac47ad3-1e72-4e81-a1ce-2194befca18c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.246710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac47ad3-1e72-4e81-a1ce-2194befca18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.271728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-0ac47ad3-1e72-4e81-a1ce-2194befca18c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:55.273099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0ac47ad3-1e72-4e81-a1ce-2194befca18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:55.288915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 948/1895] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:55.293705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac47ad3-1e72-4e81-a1ce-2194befca18c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:55.303183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac47ad3-1e72-4e81-a1ce-2194befca18c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:55.306396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4544f50-92fe-477a-a09d-a524c0a574fc admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:55.307751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0a484-4a34-4037-a391-8089d5620a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:55.308359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0a484-4a34-4037-a391-8089d5620a12 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.308637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0a484-4a34-4037-a391-8089d5620a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.308908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0a484-4a34-4037-a391-8089d5620a12 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.309634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0a484-4a34-4037-a391-8089d5620a12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:52:55.309940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0a484-4a34-4037-a391-8089d5620a12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:55.311025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:55.311536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0a484-4a34-4037-a391-8089d5620a12 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:55.311651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.311926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.312200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] PATH_INFO: `/v3/users/8eb2d88cda4443429fb381ef1de08d7a` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.313561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb2d88cda4443429fb381ef1de08d7a)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:55.315951 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:55.328670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 949/1896] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:52:55.332668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0a484-4a34-4037-a391-8089d5620a12 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:55.350090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0a484-4a34-4037-a391-8089d5620a12 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:55.355703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f687f1-8051-4ce3-9a04-25f3cb390a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:55.356507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f687f1-8051-4ce3-9a04-25f3cb390a5d admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.356789 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f687f1-8051-4ce3-9a04-25f3cb390a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.357483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f687f1-8051-4ce3-9a04-25f3cb390a5d admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.358196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f687f1-8051-4ce3-9a04-25f3cb390a5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:55.360551 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f687f1-8051-4ce3-9a04-25f3cb390a5d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:55.384083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:55.395386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb2d88cda4443429fb381ef1de08d7a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.398101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb2d88cda4443429fb381ef1de08d7a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.408290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 950/1897] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:55 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:55.409610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f687f1-8051-4ce3-9a04-25f3cb390a5d admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:55.412692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e4f687f1-8051-4ce3-9a04-25f3cb390a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:55.415162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb2d88cda4443429fb381ef1de08d7a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.424724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb2d88cda4443429fb381ef1de08d7a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.425333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb2d88cda4443429fb381ef1de08d7a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.465108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 948/1898] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:55 2022] DELETE /identity/v3/users/8eb2d88cda4443429fb381ef1de08d7a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:55.465108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:55.465108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:55.509414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f687f1-8051-4ce3-9a04-25f3cb390a5d admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:55.543692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbe0ad-b907-4df4-8c29-c4241b81c567 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:55.548774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f836eb4-a935-4903-bdbd-9cb43d32be7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:55.549524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f836eb4-a935-4903-bdbd-9cb43d32be7e admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.549920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f836eb4-a935-4903-bdbd-9cb43d32be7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.550343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f836eb4-a935-4903-bdbd-9cb43d32be7e admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.551178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f836eb4-a935-4903-bdbd-9cb43d32be7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:55.553507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f836eb4-a935-4903-bdbd-9cb43d32be7e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:55.581097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 949/1899] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:55 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:55.583047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90d5ff-1dad-49c0-bb06-661ca9c60377 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.583482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90d5ff-1dad-49c0-bb06-661ca9c60377 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.584007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90d5ff-1dad-49c0-bb06-661ca9c60377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.597453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:55.604868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-2b90d5ff-1dad-49c0-bb06-661ca9c60377 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:55.606174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2b90d5ff-1dad-49c0-bb06-661ca9c60377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:55.607247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:55.607900 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.608304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.609062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] PATH_INFO: `/v3/projects/672f2451cf0d4219b42d6bd3ffa6f473` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.610842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=672f2451cf0d4219b42d6bd3ffa6f473)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:55.613327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:55.615236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 950/1900] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:55.616308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90d5ff-1dad-49c0-bb06-661ca9c60377 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:55.627802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90d5ff-1dad-49c0-bb06-661ca9c60377 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:55.632801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169808c9-29a0-4d20-9b9e-62de09b00cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:55.633522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169808c9-29a0-4d20-9b9e-62de09b00cbe admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.633817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169808c9-29a0-4d20-9b9e-62de09b00cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.634087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169808c9-29a0-4d20-9b9e-62de09b00cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.634735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169808c9-29a0-4d20-9b9e-62de09b00cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:55.636709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169808c9-29a0-4d20-9b9e-62de09b00cbe admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:55.646874 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 951/1901] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:55.647734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169808c9-29a0-4d20-9b9e-62de09b00cbe admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:55.649730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-169808c9-29a0-4d20-9b9e-62de09b00cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:55.669806 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '672f2451cf0d4219b42d6bd3ffa6f473', 'request_id': 'req-eceb580a-caa2-4863-a86c-d2fd34cca1cf'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.680103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '672f2451cf0d4219b42d6bd3ffa6f473', 'request_id': 'req-eceb580a-caa2-4863-a86c-d2fd34cca1cf'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.688026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '672f2451cf0d4219b42d6bd3ffa6f473', 'request_id': 'req-eceb580a-caa2-4863-a86c-d2fd34cca1cf'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.702021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] The token cache is being invalidate because project 672f2451cf0d4219b42d6bd3ffa6f473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:55.702307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.702987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.704572 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 951/1902] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:55 2022] DELETE /identity/v3/projects/672f2451cf0d4219b42d6bd3ffa6f473 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:55.705408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:55.706709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:55.745408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169808c9-29a0-4d20-9b9e-62de09b00cbe admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:55.797563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb580a-caa2-4863-a86c-d2fd34cca1cf admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:55.803807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccd448-f13e-453a-b1d4-654eaa1d21dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:55.804597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccd448-f13e-453a-b1d4-654eaa1d21dd admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.805036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccd448-f13e-453a-b1d4-654eaa1d21dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.805579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccd448-f13e-453a-b1d4-654eaa1d21dd admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.806234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccd448-f13e-453a-b1d4-654eaa1d21dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:55.807866 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccd448-f13e-453a-b1d4-654eaa1d21dd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:55.816648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:55.818964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 952/1903] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:55.823278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccd448-f13e-453a-b1d4-654eaa1d21dd admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:55.824059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420f1e8-c0cf-49ae-a9a8-aea6cf73840e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:55.825064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-49ccd448-f13e-453a-b1d4-654eaa1d21dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:55.825232 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420f1e8-c0cf-49ae-a9a8-aea6cf73840e service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.825838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420f1e8-c0cf-49ae-a9a8-aea6cf73840e service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.826812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420f1e8-c0cf-49ae-a9a8-aea6cf73840e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.829042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8420f1e8-c0cf-49ae-a9a8-aea6cf73840e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:55.840715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccd448-f13e-453a-b1d4-654eaa1d21dd admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:55.845995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:55.846636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.846914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.847187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] PATH_INFO: `/v3/users/244c2b2f81e444e98f27eef0dea0e6d0` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.848843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=244c2b2f81e444e98f27eef0dea0e6d0)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:55.850268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:55.850860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420f1e8-c0cf-49ae-a9a8-aea6cf73840e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:55.852426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420f1e8-c0cf-49ae-a9a8-aea6cf73840e service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:55.863792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 952/1904] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:55.864615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420f1e8-c0cf-49ae-a9a8-aea6cf73840e service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:55.873934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420f1e8-c0cf-49ae-a9a8-aea6cf73840e service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:55.881398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032f95a-849b-4b7b-95ca-1b14f907b29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:55.884237 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032f95a-849b-4b7b-95ca-1b14f907b29c admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.884516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032f95a-849b-4b7b-95ca-1b14f907b29c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.884889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032f95a-849b-4b7b-95ca-1b14f907b29c admin admin] PATH_INFO: `/v3/projects/2a3f1ef0d0c343dfbf2eae49c0ea450c/users/fc880a31f5be4ea2a500bfa504749ce0/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.916267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032f95a-849b-4b7b-95ca-1b14f907b29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a3f1ef0d0c343dfbf2eae49c0ea450c, user_id=fc880a31f5be4ea2a500bfa504749ce0, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:55.919259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032f95a-849b-4b7b-95ca-1b14f907b29c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:55.926281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '244c2b2f81e444e98f27eef0dea0e6d0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.926820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '244c2b2f81e444e98f27eef0dea0e6d0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.938099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '244c2b2f81e444e98f27eef0dea0e6d0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.939453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 953/1905] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:55 2022] PUT /identity/v3/projects/2a3f1ef0d0c343dfbf2eae49c0ea450c/users/fc880a31f5be4ea2a500bfa504749ce0/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:55.950306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79566a8-d3de-46b3-a43b-87b83a480cf1 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:55.950882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79566a8-d3de-46b3-a43b-87b83a480cf1 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:55.951253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79566a8-d3de-46b3-a43b-87b83a480cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:55.961965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '244c2b2f81e444e98f27eef0dea0e6d0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.963917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:55.973108 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '244c2b2f81e444e98f27eef0dea0e6d0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:55.981659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 953/1906] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:55 2022] DELETE /identity/v3/users/244c2b2f81e444e98f27eef0dea0e6d0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:55.988252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f79566a8-d3de-46b3-a43b-87b83a480cf1 None None] MFA Rules not processed for user `fc880a31f5be4ea2a500bfa504749ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:55.989668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f79566a8-d3de-46b3-a43b-87b83a480cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:56.088251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 954/1907] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:56.101161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:56.102797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:56.173333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249b8af-0b51-4e57-9396-a3cbe96e4a37 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:56.177853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69158e0f-1768-4c09-a17b-48c129408b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:56.178535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69158e0f-1768-4c09-a17b-48c129408b37 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:56.178871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69158e0f-1768-4c09-a17b-48c129408b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:56.179148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69158e0f-1768-4c09-a17b-48c129408b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:56.192112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69158e0f-1768-4c09-a17b-48c129408b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:56.193906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69158e0f-1768-4c09-a17b-48c129408b37 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:56.204774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 954/1908] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:56.333376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79566a8-d3de-46b3-a43b-87b83a480cf1 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:56.335147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f79566a8-d3de-46b3-a43b-87b83a480cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:56.383557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69158e0f-1768-4c09-a17b-48c129408b37 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:56.385981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-69158e0f-1768-4c09-a17b-48c129408b37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:56.451414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79566a8-d3de-46b3-a43b-87b83a480cf1 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:56.457340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:56.458117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:56.458388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:56.458875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] PATH_INFO: `/v3/projects/207889e387e740279c8a957c6b0b7362` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:56.460320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=207889e387e740279c8a957c6b0b7362)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:56.462549 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:56.473986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69158e0f-1768-4c09-a17b-48c129408b37 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:56.481448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:56.482641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:56.483067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:56.483573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] PATH_INFO: `/v3/projects/7853ff311f20431399e382138f6e391d` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:56.485388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] RBAC: Authorizing `identity:delete_project(project_id=7853ff311f20431399e382138f6e391d)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:56.488792 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:56.525692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '207889e387e740279c8a957c6b0b7362', 'request_id': 'req-91c7dbf5-813a-4fff-9ad2-76fedddf6548'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.537309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '207889e387e740279c8a957c6b0b7362', 'request_id': 'req-91c7dbf5-813a-4fff-9ad2-76fedddf6548'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.538666 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:56.548006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '207889e387e740279c8a957c6b0b7362', 'request_id': 'req-91c7dbf5-813a-4fff-9ad2-76fedddf6548'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.561631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7853ff311f20431399e382138f6e391d', 'request_id': 'req-9f63f46d-2791-4657-8fa8-90c097b04bda'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.564402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] The token cache is being invalidate because project 207889e387e740279c8a957c6b0b7362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:56.565297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.566007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.568099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 955/1909] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:56 2022] DELETE /identity/v3/projects/207889e387e740279c8a957c6b0b7362 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:56.572692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7853ff311f20431399e382138f6e391d', 'request_id': 'req-9f63f46d-2791-4657-8fa8-90c097b04bda'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.572838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:56.577561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:56.582405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7853ff311f20431399e382138f6e391d', 'request_id': 'req-9f63f46d-2791-4657-8fa8-90c097b04bda'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.602952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] The token cache is being invalidate because project 7853ff311f20431399e382138f6e391d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:56.603247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.603902 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9f63f46d-2791-4657-8fa8-90c097b04bda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.613560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 955/1910] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:56 2022] DELETE /identity/v3/projects/7853ff311f20431399e382138f6e391d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:56.615680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae992bc-9980-4982-b9c2-85c03d3382a6 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:56.616024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae992bc-9980-4982-b9c2-85c03d3382a6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:56.616363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae992bc-9980-4982-b9c2-85c03d3382a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:56.637729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9ae992bc-9980-4982-b9c2-85c03d3382a6 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:56.639008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9ae992bc-9980-4982-b9c2-85c03d3382a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:56.659767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:56.671002 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7dbf5-813a-4fff-9ad2-76fedddf6548 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:56.709145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 956/1911] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:56.717339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db569567-2f50-4de2-a80a-10049368ba0e None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:56.717662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db569567-2f50-4de2-a80a-10049368ba0e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:56.718192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db569567-2f50-4de2-a80a-10049368ba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:56.739573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-db569567-2f50-4de2-a80a-10049368ba0e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:56.740881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-db569567-2f50-4de2-a80a-10049368ba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:56.753806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:56.754358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 957/1912] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:56.754833 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:56.754949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db569567-2f50-4de2-a80a-10049368ba0e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:56.755246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:56.755668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] PATH_INFO: `/v3/users/36d46c9c38b24d4ebb8dda86b28dbecc` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:56.757864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d46c9c38b24d4ebb8dda86b28dbecc)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:56.763486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:56.766088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db569567-2f50-4de2-a80a-10049368ba0e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:56.772326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cabc13-3c9e-43b2-9e3d-9c7d3db95c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:56.773828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cabc13-3c9e-43b2-9e3d-9c7d3db95c3e admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:56.774201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cabc13-3c9e-43b2-9e3d-9c7d3db95c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:56.774472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cabc13-3c9e-43b2-9e3d-9c7d3db95c3e admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:56.775152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cabc13-3c9e-43b2-9e3d-9c7d3db95c3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:52:56.775456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cabc13-3c9e-43b2-9e3d-9c7d3db95c3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:56.777833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cabc13-3c9e-43b2-9e3d-9c7d3db95c3e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:56.797189 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 958/1913] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:52:56.798244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cabc13-3c9e-43b2-9e3d-9c7d3db95c3e admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:56.821107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cabc13-3c9e-43b2-9e3d-9c7d3db95c3e admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:56.828015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bf13a-9aba-4bf7-a246-8d9561be7048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:56.828933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bf13a-9aba-4bf7-a246-8d9561be7048 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:56.834386 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bf13a-9aba-4bf7-a246-8d9561be7048 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:56.837996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bf13a-9aba-4bf7-a246-8d9561be7048 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:56.838893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bf13a-9aba-4bf7-a246-8d9561be7048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:56.841580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bf13a-9aba-4bf7-a246-8d9561be7048 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:56.866642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d46c9c38b24d4ebb8dda86b28dbecc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.867491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d46c9c38b24d4ebb8dda86b28dbecc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.876188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d46c9c38b24d4ebb8dda86b28dbecc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.881078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 959/1914] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:56 2022] POST /identity/v3/projects => generated 365 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:56.884837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bf13a-9aba-4bf7-a246-8d9561be7048 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:56.905141 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d46c9c38b24d4ebb8dda86b28dbecc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.906044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d46c9c38b24d4ebb8dda86b28dbecc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:56.927167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 956/1915] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:52:56 2022] DELETE /identity/v3/users/36d46c9c38b24d4ebb8dda86b28dbecc => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:56.974016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bf13a-9aba-4bf7-a246-8d9561be7048 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:57.046553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc285021-a628-4b26-a38b-a1df791111ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:57.048319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc285021-a628-4b26-a38b-a1df791111ed admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.049044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc285021-a628-4b26-a38b-a1df791111ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.049845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc285021-a628-4b26-a38b-a1df791111ed admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.051036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc285021-a628-4b26-a38b-a1df791111ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:57.053336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc285021-a628-4b26-a38b-a1df791111ed admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:57.093219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 960/1916] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:56 2022] POST /identity/v3/users => generated 283 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:57.095500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:57.097215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:57.112713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376f699-9ce6-4362-b499-abc0c42cc4c6 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:57.118821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33dddf-7e44-4d17-a63e-3142ef64ed18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:57.120181 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33dddf-7e44-4d17-a63e-3142ef64ed18 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.120561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33dddf-7e44-4d17-a63e-3142ef64ed18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.120910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33dddf-7e44-4d17-a63e-3142ef64ed18 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.121669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33dddf-7e44-4d17-a63e-3142ef64ed18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:57.123386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33dddf-7e44-4d17-a63e-3142ef64ed18 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:57.130027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:57.138811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 957/1917] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:57.142888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc285021-a628-4b26-a38b-a1df791111ed admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:57.164555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc285021-a628-4b26-a38b-a1df791111ed admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:57.194769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905bf68-1cd2-469e-bfb7-573ce76d4172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:57.195776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905bf68-1cd2-469e-bfb7-573ce76d4172 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.196245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905bf68-1cd2-469e-bfb7-573ce76d4172 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.197183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905bf68-1cd2-469e-bfb7-573ce76d4172 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.198435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905bf68-1cd2-469e-bfb7-573ce76d4172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:57.201401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905bf68-1cd2-469e-bfb7-573ce76d4172 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:57.220846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 961/1918] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:57.225958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33dddf-7e44-4d17-a63e-3142ef64ed18 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:57.257532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33dddf-7e44-4d17-a63e-3142ef64ed18 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:57.264014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ea602-947d-43d8-a63d-a39a8f1cba43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:57.265244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ea602-947d-43d8-a63d-a39a8f1cba43 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.265244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ea602-947d-43d8-a63d-a39a8f1cba43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.265431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ea602-947d-43d8-a63d-a39a8f1cba43 admin admin] PATH_INFO: `/v3/projects/9c9d0ddc22504ceba5307e5936f2585f/users/269725f47ebe405193acaa4975d3c4c4/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.289220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ea602-947d-43d8-a63d-a39a8f1cba43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9d0ddc22504ceba5307e5936f2585f, user_id=269725f47ebe405193acaa4975d3c4c4, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:57.292451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ea602-947d-43d8-a63d-a39a8f1cba43 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:57.328383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 958/1919] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:57 2022] PUT /identity/v3/projects/9c9d0ddc22504ceba5307e5936f2585f/users/269725f47ebe405193acaa4975d3c4c4/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:57.343621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c6dec-4f5a-476f-8114-51084017c871 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.344334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c6dec-4f5a-476f-8114-51084017c871 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.344851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c6dec-4f5a-476f-8114-51084017c871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.377594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:57.389880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-1e6c6dec-4f5a-476f-8114-51084017c871 None None] MFA Rules not processed for user `269725f47ebe405193acaa4975d3c4c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:57.391623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1e6c6dec-4f5a-476f-8114-51084017c871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:57.514993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 962/1920] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:57.532699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ea602-947d-43d8-a63d-a39a8f1cba43 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:57.541960 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c6dec-4f5a-476f-8114-51084017c871 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:57.543709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1e6c6dec-4f5a-476f-8114-51084017c871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:57.618620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ea602-947d-43d8-a63d-a39a8f1cba43 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:57.628571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578e2af-14e3-4bad-9daa-f19c9904ee12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:57.630227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578e2af-14e3-4bad-9daa-f19c9904ee12 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.630602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578e2af-14e3-4bad-9daa-f19c9904ee12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.630941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578e2af-14e3-4bad-9daa-f19c9904ee12 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.631844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578e2af-14e3-4bad-9daa-f19c9904ee12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:57.634850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578e2af-14e3-4bad-9daa-f19c9904ee12 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:57.637659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c6dec-4f5a-476f-8114-51084017c871 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:57.643461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:57.644525 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.645560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.645851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] PATH_INFO: `/v3/projects/70e383c5d71e442686570ab1428d47fe` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.647150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e383c5d71e442686570ab1428d47fe)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:57.649543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:57.685703 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 959/1921] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:57 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:57.686611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578e2af-14e3-4bad-9daa-f19c9904ee12 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:57.721165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e383c5d71e442686570ab1428d47fe', 'request_id': 'req-c4f951aa-ea21-4446-8af2-5d2969e412d9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:57.730267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e383c5d71e442686570ab1428d47fe', 'request_id': 'req-c4f951aa-ea21-4446-8af2-5d2969e412d9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:57.738823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e383c5d71e442686570ab1428d47fe', 'request_id': 'req-c4f951aa-ea21-4446-8af2-5d2969e412d9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:57.751334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] The token cache is being invalidate because project 70e383c5d71e442686570ab1428d47fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:52:57.751973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:57.753225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c4f951aa-ea21-4446-8af2-5d2969e412d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:52:57.755630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 963/1922] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:52:57 2022] DELETE /identity/v3/projects/70e383c5d71e442686570ab1428d47fe => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:52:57.757928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59df72-e672-4898-afde-3e1200ebfab1 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.758643 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59df72-e672-4898-afde-3e1200ebfab1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.759093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59df72-e672-4898-afde-3e1200ebfab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.785390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ec59df72-e672-4898-afde-3e1200ebfab1 None None] MFA Rules not processed for user `fc880a31f5be4ea2a500bfa504749ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:57.786714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ec59df72-e672-4898-afde-3e1200ebfab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:57.798722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578e2af-14e3-4bad-9daa-f19c9904ee12 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:57.814063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:57.863245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 964/1923] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:57.870923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6714b-aaa3-4974-be73-ba028a793889 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.871226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6714b-aaa3-4974-be73-ba028a793889 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.871572 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6714b-aaa3-4974-be73-ba028a793889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.876081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c955c-f9f0-4199-9fb6-62d6a7cc2fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:57.877285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c955c-f9f0-4199-9fb6-62d6a7cc2fcc admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.877598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c955c-f9f0-4199-9fb6-62d6a7cc2fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.877873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c955c-f9f0-4199-9fb6-62d6a7cc2fcc admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.878509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c955c-f9f0-4199-9fb6-62d6a7cc2fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:57.881909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c955c-f9f0-4199-9fb6-62d6a7cc2fcc admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:57.896647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b6a6714b-aaa3-4974-be73-ba028a793889 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:57.898482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b6a6714b-aaa3-4974-be73-ba028a793889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:57.904730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:57.915410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 960/1924] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:57 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:57.917647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 965/1925] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:57.923030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75960a97-d4ec-4c72-bce9-6545460df033 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.923320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75960a97-d4ec-4c72-bce9-6545460df033 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.923596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75960a97-d4ec-4c72-bce9-6545460df033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.931894 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6714b-aaa3-4974-be73-ba028a793889 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:57.935878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b6a6714b-aaa3-4974-be73-ba028a793889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:57.951701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6714b-aaa3-4974-be73-ba028a793889 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:57.952462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-75960a97-d4ec-4c72-bce9-6545460df033 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:57.954166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-75960a97-d4ec-4c72-bce9-6545460df033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:57.957451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847ac3c-95a9-4900-884a-123a391f07ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:57.958087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847ac3c-95a9-4900-884a-123a391f07ea admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.958363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847ac3c-95a9-4900-884a-123a391f07ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.958670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847ac3c-95a9-4900-884a-123a391f07ea admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.959325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847ac3c-95a9-4900-884a-123a391f07ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:57.960968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847ac3c-95a9-4900-884a-123a391f07ea admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:57.965023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 961/1926] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:57.969354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75960a97-d4ec-4c72-bce9-6545460df033 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:57.973998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 966/1927] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:57.977442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847ac3c-95a9-4900-884a-123a391f07ea admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:57.983564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75960a97-d4ec-4c72-bce9-6545460df033 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:57.988203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847ac3c-95a9-4900-884a-123a391f07ea admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:57.989180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017226e2-6b92-45bf-9198-2d2cbdb352e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:57.990023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017226e2-6b92-45bf-9198-2d2cbdb352e2 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.990585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017226e2-6b92-45bf-9198-2d2cbdb352e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.991045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017226e2-6b92-45bf-9198-2d2cbdb352e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:57.991879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017226e2-6b92-45bf-9198-2d2cbdb352e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:52:57.992179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017226e2-6b92-45bf-9198-2d2cbdb352e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:57.994117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017226e2-6b92-45bf-9198-2d2cbdb352e2 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:57.994907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d2146-bed8-41b9-9c7d-0556a8354ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:57.995825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d2146-bed8-41b9-9c7d-0556a8354ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:57.996224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d2146-bed8-41b9-9c7d-0556a8354ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:57.996565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d2146-bed8-41b9-9c7d-0556a8354ae8 admin admin] PATH_INFO: `/v3/projects/201513664f904312b7e73c65ab84aba5/users/5500bc5f3e184a60ba06e1c9a970b250/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.010828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 962/1928] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:52:58.012269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017226e2-6b92-45bf-9198-2d2cbdb352e2 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.023199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d2146-bed8-41b9-9c7d-0556a8354ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201513664f904312b7e73c65ab84aba5, user_id=5500bc5f3e184a60ba06e1c9a970b250, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.025223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017226e2-6b92-45bf-9198-2d2cbdb352e2 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:58.026939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d2146-bed8-41b9-9c7d-0556a8354ae8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.030206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532c910-deac-4cd5-831b-df6378d466fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:58.031889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532c910-deac-4cd5-831b-df6378d466fb admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.031889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532c910-deac-4cd5-831b-df6378d466fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.032068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532c910-deac-4cd5-831b-df6378d466fb admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.032602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532c910-deac-4cd5-831b-df6378d466fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.035291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532c910-deac-4cd5-831b-df6378d466fb admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.049561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 967/1929] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:57 2022] PUT /identity/v3/projects/201513664f904312b7e73c65ab84aba5/users/5500bc5f3e184a60ba06e1c9a970b250/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:58.055319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d2146-bed8-41b9-9c7d-0556a8354ae8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.077928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 963/1930] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:58 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:52:58.080720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532c910-deac-4cd5-831b-df6378d466fb admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.138951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d2146-bed8-41b9-9c7d-0556a8354ae8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:58.179201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532c910-deac-4cd5-831b-df6378d466fb admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:58.185533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d92e-5aeb-4c94-bee2-6e7e748d2b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:58.186756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d92e-5aeb-4c94-bee2-6e7e748d2b70 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.186756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d92e-5aeb-4c94-bee2-6e7e748d2b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.187101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d92e-5aeb-4c94-bee2-6e7e748d2b70 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.187817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06d92e-5aeb-4c94-bee2-6e7e748d2b70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.190424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06d92e-5aeb-4c94-bee2-6e7e748d2b70 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.220927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 964/1931] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:58 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:52:58.225218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d92e-5aeb-4c94-bee2-6e7e748d2b70 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.247574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d92e-5aeb-4c94-bee2-6e7e748d2b70 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:58.250552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930932e-16d6-492a-a446-886de0d1a27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:58.251156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930932e-16d6-492a-a446-886de0d1a27a admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.251439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930932e-16d6-492a-a446-886de0d1a27a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.251715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930932e-16d6-492a-a446-886de0d1a27a admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.252952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930932e-16d6-492a-a446-886de0d1a27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.254649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930932e-16d6-492a-a446-886de0d1a27a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.261834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b62bd9-20bc-4521-b1fe-c20084478f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:58.262561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62bd9-20bc-4521-b1fe-c20084478f34 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.262837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62bd9-20bc-4521-b1fe-c20084478f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.263167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62bd9-20bc-4521-b1fe-c20084478f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.263832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b62bd9-20bc-4521-b1fe-c20084478f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.265772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b62bd9-20bc-4521-b1fe-c20084478f34 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.280489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 965/1932] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:58.285745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b62bd9-20bc-4521-b1fe-c20084478f34 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.286172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 968/1933] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:58.287100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930932e-16d6-492a-a446-886de0d1a27a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.298311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930932e-16d6-492a-a446-886de0d1a27a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:58.303505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b62bd9-20bc-4521-b1fe-c20084478f34 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:58.310537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3b0d5-215e-4615-a8ce-b41765b8d23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:58.312203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3b0d5-215e-4615-a8ce-b41765b8d23a admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.312491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3b0d5-215e-4615-a8ce-b41765b8d23a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.312999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3b0d5-215e-4615-a8ce-b41765b8d23a admin admin] PATH_INFO: `/v3/projects/201513664f904312b7e73c65ab84aba5/users/5500bc5f3e184a60ba06e1c9a970b250/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.315639 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3b0d5-215e-4615-a8ce-b41765b8d23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=201513664f904312b7e73c65ab84aba5, user_id=5500bc5f3e184a60ba06e1c9a970b250, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.320625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75dc1e2-b233-4459-b110-b83388b1a093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:58.321358 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dc1e2-b233-4459-b110-b83388b1a093 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.321623 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dc1e2-b233-4459-b110-b83388b1a093 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.321816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dc1e2-b233-4459-b110-b83388b1a093 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.322241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3b0d5-215e-4615-a8ce-b41765b8d23a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.322459 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75dc1e2-b233-4459-b110-b83388b1a093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.324379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75dc1e2-b233-4459-b110-b83388b1a093 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.339170 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 969/1934] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:58 2022] PUT /identity/v3/projects/201513664f904312b7e73c65ab84aba5/users/5500bc5f3e184a60ba06e1c9a970b250/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:58.342542 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 966/1935] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:58.349812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3b0d5-215e-4615-a8ce-b41765b8d23a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.351591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dfef4-8f8f-4d31-b6ce-9c366507cd06 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.351873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dfef4-8f8f-4d31-b6ce-9c366507cd06 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.352140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dfef4-8f8f-4d31-b6ce-9c366507cd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.388048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:58.395709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-969dfef4-8f8f-4d31-b6ce-9c366507cd06 None None] MFA Rules not processed for user `5500bc5f3e184a60ba06e1c9a970b250`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:58.397616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-969dfef4-8f8f-4d31-b6ce-9c366507cd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:58.441401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3b0d5-215e-4615-a8ce-b41765b8d23a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:58.446570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbfccd-b9b4-412a-af7c-f4dfe9d29be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:58.447385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbfccd-b9b4-412a-af7c-f4dfe9d29be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.447744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbfccd-b9b4-412a-af7c-f4dfe9d29be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.448118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbfccd-b9b4-412a-af7c-f4dfe9d29be9 admin admin] PATH_INFO: `/v3/projects/4d1bb3f5687f47c18d4a75601d92edd6/users/6ec2e8bd8f184e6fa954916cd25e203d/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.465710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbfccd-b9b4-412a-af7c-f4dfe9d29be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1bb3f5687f47c18d4a75601d92edd6, user_id=6ec2e8bd8f184e6fa954916cd25e203d, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.470142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbfccd-b9b4-412a-af7c-f4dfe9d29be9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.499186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 970/1936] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:58 2022] PUT /identity/v3/projects/4d1bb3f5687f47c18d4a75601d92edd6/users/6ec2e8bd8f184e6fa954916cd25e203d/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:58.500784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23100ea1-4c57-4a5b-bb23-7876c53c4a4d None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.501080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23100ea1-4c57-4a5b-bb23-7876c53c4a4d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.501364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23100ea1-4c57-4a5b-bb23-7876c53c4a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.506398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 967/1937] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:58.517264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dfef4-8f8f-4d31-b6ce-9c366507cd06 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.518995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-969dfef4-8f8f-4d31-b6ce-9c366507cd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:58.530746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:58.541206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-23100ea1-4c57-4a5b-bb23-7876c53c4a4d None None] MFA Rules not processed for user `6ec2e8bd8f184e6fa954916cd25e203d`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:58.542776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-23100ea1-4c57-4a5b-bb23-7876c53c4a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:58.604286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dfef4-8f8f-4d31-b6ce-9c366507cd06 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:58.668567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 971/1938] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:52:58.674800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23100ea1-4c57-4a5b-bb23-7876c53c4a4d None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.696913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1ee2d-eab0-450c-a5d4-0aba66c1de11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:58.697913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1ee2d-eab0-450c-a5d4-0aba66c1de11 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.698198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1ee2d-eab0-450c-a5d4-0aba66c1de11 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.698507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1ee2d-eab0-450c-a5d4-0aba66c1de11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.788231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1ee2d-eab0-450c-a5d4-0aba66c1de11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.789760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1ee2d-eab0-450c-a5d4-0aba66c1de11 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.801138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 968/1939] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:58.801825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23100ea1-4c57-4a5b-bb23-7876c53c4a4d None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:58.814855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221db1d2-bddb-43d0-9cac-771338f03c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:58.815679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221db1d2-bddb-43d0-9cac-771338f03c00 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.815968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221db1d2-bddb-43d0-9cac-771338f03c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.816245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221db1d2-bddb-43d0-9cac-771338f03c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.818510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221db1d2-bddb-43d0-9cac-771338f03c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.820513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221db1d2-bddb-43d0-9cac-771338f03c00 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.873130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 972/1940] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:52:58 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:58.877317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1ee2d-eab0-450c-a5d4-0aba66c1de11 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.897459 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1ee2d-eab0-450c-a5d4-0aba66c1de11 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:58.962310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221db1d2-bddb-43d0-9cac-771338f03c00 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:58.987320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74fb7a-cc2a-4af6-a4ea-870250198a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:58.987951 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74fb7a-cc2a-4af6-a4ea-870250198a89 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:58.988227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74fb7a-cc2a-4af6-a4ea-870250198a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:58.988497 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74fb7a-cc2a-4af6-a4ea-870250198a89 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:58.989133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74fb7a-cc2a-4af6-a4ea-870250198a89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:58.991731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74fb7a-cc2a-4af6-a4ea-870250198a89 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:58.992317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:59.034808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 969/1941] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:52:58 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:52:59.037044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74fb7a-cc2a-4af6-a4ea-870250198a89 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:59.045170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:59.052187 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74fb7a-cc2a-4af6-a4ea-870250198a89 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:59.060638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6354-3092-4f6d-9069-2420434af98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:59.061267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6354-3092-4f6d-9069-2420434af98e admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:59.061686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6354-3092-4f6d-9069-2420434af98e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:59.061960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6354-3092-4f6d-9069-2420434af98e admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:59.062680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e6354-3092-4f6d-9069-2420434af98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:59.065398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e6354-3092-4f6d-9069-2420434af98e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:59.066735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221db1d2-bddb-43d0-9cac-771338f03c00 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:59.070641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b84988-9e80-4c9d-8a43-ec853a68cb07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:59.071261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b84988-9e80-4c9d-8a43-ec853a68cb07 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:59.071556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b84988-9e80-4c9d-8a43-ec853a68cb07 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:59.071830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b84988-9e80-4c9d-8a43-ec853a68cb07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:59.073667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-74b84988-9e80-4c9d-8a43-ec853a68cb07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:59.106409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 970/1942] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:52:59.108077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6354-3092-4f6d-9069-2420434af98e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:59.141639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6354-3092-4f6d-9069-2420434af98e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:59.156519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514b61f-f257-4671-acfb-7c4d2542dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:59.159139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514b61f-f257-4671-acfb-7c4d2542dd4f admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:59.159421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514b61f-f257-4671-acfb-7c4d2542dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:59.159695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514b61f-f257-4671-acfb-7c4d2542dd4f admin admin] PATH_INFO: `/v3/projects/b69a73d905f9451288dbba24ea46725e/users/7270a704f73f4e4a837925d669c903a0/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:59.187732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b84988-9e80-4c9d-8a43-ec853a68cb07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:59.189218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b84988-9e80-4c9d-8a43-ec853a68cb07 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:59.197371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514b61f-f257-4671-acfb-7c4d2542dd4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69a73d905f9451288dbba24ea46725e, user_id=7270a704f73f4e4a837925d669c903a0, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:59.201083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514b61f-f257-4671-acfb-7c4d2542dd4f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:59.202144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 973/1943] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:52:59.223378 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 971/1944] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:59 2022] PUT /identity/v3/projects/b69a73d905f9451288dbba24ea46725e/users/7270a704f73f4e4a837925d669c903a0/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:59.227634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b84988-9e80-4c9d-8a43-ec853a68cb07 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:59.240295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b84988-9e80-4c9d-8a43-ec853a68cb07 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:59.359762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc8ac0-ccd6-4f7d-863e-f982cfe3ca4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:59.361053 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc8ac0-ccd6-4f7d-863e-f982cfe3ca4c admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:59.362352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc8ac0-ccd6-4f7d-863e-f982cfe3ca4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:59.363034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc8ac0-ccd6-4f7d-863e-f982cfe3ca4c admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:59.363359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc8ac0-ccd6-4f7d-863e-f982cfe3ca4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:59.364978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc8ac0-ccd6-4f7d-863e-f982cfe3ca4c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:59.380924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 974/1945] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:52:59.382145 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514b61f-f257-4671-acfb-7c4d2542dd4f admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:59.393599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514b61f-f257-4671-acfb-7c4d2542dd4f admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:59.398901 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7b038-ae75-4d68-9205-b62495716b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:59.399563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7b038-ae75-4d68-9205-b62495716b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:59.399839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7b038-ae75-4d68-9205-b62495716b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:59.400108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7b038-ae75-4d68-9205-b62495716b50 admin admin] PATH_INFO: `/v3/projects/b69a73d905f9451288dbba24ea46725e/users/7270a704f73f4e4a837925d669c903a0/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:59.402151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7b038-ae75-4d68-9205-b62495716b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69a73d905f9451288dbba24ea46725e, user_id=7270a704f73f4e4a837925d669c903a0, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:59.415720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7b038-ae75-4d68-9205-b62495716b50 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:59.430452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 972/1946] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:52:59 2022] PUT /identity/v3/projects/b69a73d905f9451288dbba24ea46725e/users/7270a704f73f4e4a837925d669c903a0/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:52:59.443608 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e7e46-dfa6-4860-8581-4017343ae3a7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:59.443887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e7e46-dfa6-4860-8581-4017343ae3a7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:59.444168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e7e46-dfa6-4860-8581-4017343ae3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:59.484321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9b2e7e46-dfa6-4860-8581-4017343ae3a7 None None] MFA Rules not processed for user `7270a704f73f4e4a837925d669c903a0`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:52:59.487154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9b2e7e46-dfa6-4860-8581-4017343ae3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:52:59.588610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:52:59.620120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 975/1947] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:52:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:52:59.640440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7b038-ae75-4d68-9205-b62495716b50 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:59.751884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7b038-ae75-4d68-9205-b62495716b50 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:59.756152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42725ffe-e538-4b22-b0d0-48fa79331dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:59.756859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42725ffe-e538-4b22-b0d0-48fa79331dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:59.757204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42725ffe-e538-4b22-b0d0-48fa79331dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:59.757569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42725ffe-e538-4b22-b0d0-48fa79331dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:59.776505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42725ffe-e538-4b22-b0d0-48fa79331dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:59.778090 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42725ffe-e538-4b22-b0d0-48fa79331dc1 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:59.788620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 973/1948] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:52:59.868547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e7e46-dfa6-4860-8581-4017343ae3a7 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:52:59.882520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e7e46-dfa6-4860-8581-4017343ae3a7 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:52:59.886476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7e429-eea4-4403-90b3-e94322de6c93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:52:59.887102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e429-eea4-4403-90b3-e94322de6c93 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:52:59.887379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e429-eea4-4403-90b3-e94322de6c93 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:52:59.887649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e429-eea4-4403-90b3-e94322de6c93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:52:59.947682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7e429-eea4-4403-90b3-e94322de6c93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:52:59.949079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7e429-eea4-4403-90b3-e94322de6c93 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:52:59.958805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 976/1949] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:52:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:07.671349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42725ffe-e538-4b22-b0d0-48fa79331dc1 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:07.731649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42725ffe-e538-4b22-b0d0-48fa79331dc1 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:07.736653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:07.737287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:07.737604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:07.737878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] PATH_INFO: `/v3/users/269725f47ebe405193acaa4975d3c4c4` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:07.738996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=269725f47ebe405193acaa4975d3c4c4)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:07.740941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:07.785120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269725f47ebe405193acaa4975d3c4c4'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:07.785787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269725f47ebe405193acaa4975d3c4c4'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:07.792539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269725f47ebe405193acaa4975d3c4c4'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:07.796055 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:07.800143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269725f47ebe405193acaa4975d3c4c4'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:07.800815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269725f47ebe405193acaa4975d3c4c4'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:07.818236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 974/1950] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:07 2022] DELETE /identity/v3/users/269725f47ebe405193acaa4975d3c4c4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:07.822692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218b60-4c2a-40aa-9741-c68ed3d80117 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:07.823162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218b60-4c2a-40aa-9741-c68ed3d80117 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:07.823294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218b60-4c2a-40aa-9741-c68ed3d80117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:07.842437 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-12218b60-4c2a-40aa-9741-c68ed3d80117 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:07.843863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-12218b60-4c2a-40aa-9741-c68ed3d80117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:07.902706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 977/1951] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:07.912819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:07.914236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:07.970992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9903c5-ee51-4a4e-a4b2-c26d2d362abd admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:07.975749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd69b1-d96e-47ef-af2f-5703a98332ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:07.976356 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd69b1-d96e-47ef-af2f-5703a98332ba service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:07.976631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd69b1-d96e-47ef-af2f-5703a98332ba service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:07.976898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd69b1-d96e-47ef-af2f-5703a98332ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:07.987638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd69b1-d96e-47ef-af2f-5703a98332ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:07.989115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd69b1-d96e-47ef-af2f-5703a98332ba service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:08.000065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 975/1952] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:08.922858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12218b60-4c2a-40aa-9741-c68ed3d80117 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:08.925861 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-12218b60-4c2a-40aa-9741-c68ed3d80117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:08.943614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12218b60-4c2a-40aa-9741-c68ed3d80117 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:08.950775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:08.951467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:08.951737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:08.952042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] PATH_INFO: `/v3/projects/9c9d0ddc22504ceba5307e5936f2585f` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:08.953318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9d0ddc22504ceba5307e5936f2585f)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:08.955698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:09.028825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9d0ddc22504ceba5307e5936f2585f', 'request_id': 'req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.038922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9d0ddc22504ceba5307e5936f2585f', 'request_id': 'req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.048939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9d0ddc22504ceba5307e5936f2585f', 'request_id': 'req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.061034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] The token cache is being invalidate because project 9c9d0ddc22504ceba5307e5936f2585f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:09.061519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.062249 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.065289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 978/1953] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:08 2022] DELETE /identity/v3/projects/9c9d0ddc22504ceba5307e5936f2585f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:09.069283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd69b1-d96e-47ef-af2f-5703a98332ba service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:09.079921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd69b1-d96e-47ef-af2f-5703a98332ba service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:09.143777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:09.144404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:09.144682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:09.144953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] PATH_INFO: `/v3/users/5500bc5f3e184a60ba06e1c9a970b250` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:09.146088 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5500bc5f3e184a60ba06e1c9a970b250)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:09.148022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:09.193090 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5500bc5f3e184a60ba06e1c9a970b250'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.193705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5500bc5f3e184a60ba06e1c9a970b250'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.200662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5500bc5f3e184a60ba06e1c9a970b250'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.207728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5500bc5f3e184a60ba06e1c9a970b250'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.208430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-621e3cff-f1e1-482a-b663-7bddba3fbf47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5500bc5f3e184a60ba06e1c9a970b250'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.222602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:09.227209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 976/1954] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:09 2022] DELETE /identity/v3/users/5500bc5f3e184a60ba06e1c9a970b250 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:53:09.572173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:09.573715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:09.587850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3016a-e5ca-4bef-ae7f-fbb8d8636964 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:09.602141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:09.649051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:09.649723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:09.650000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:09.650270 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] PATH_INFO: `/v3/projects/201513664f904312b7e73c65ab84aba5` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:09.651404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=201513664f904312b7e73c65ab84aba5)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:09.653301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:09.703826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201513664f904312b7e73c65ab84aba5', 'request_id': 'req-9fbf2370-6e80-4280-ad6d-40b60da9fb93'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.710813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201513664f904312b7e73c65ab84aba5', 'request_id': 'req-9fbf2370-6e80-4280-ad6d-40b60da9fb93'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.717277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201513664f904312b7e73c65ab84aba5', 'request_id': 'req-9fbf2370-6e80-4280-ad6d-40b60da9fb93'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.728038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] The token cache is being invalidate because project 201513664f904312b7e73c65ab84aba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:09.728331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.728970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9fbf2370-6e80-4280-ad6d-40b60da9fb93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:09.730484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 979/1955] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:09 2022] DELETE /identity/v3/projects/201513664f904312b7e73c65ab84aba5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:09.735567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b2a7-79d6-4fef-863c-ca3b494339a7 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:09.735915 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b2a7-79d6-4fef-863c-ca3b494339a7 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:09.736192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b2a7-79d6-4fef-863c-ca3b494339a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:09.756022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f5c3b2a7-79d6-4fef-863c-ca3b494339a7 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:09.757222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f5c3b2a7-79d6-4fef-863c-ca3b494339a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:09.814637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 977/1956] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:09.823620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c066c-4385-4c91-9d0f-9e31791c5109 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:09.823933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c066c-4385-4c91-9d0f-9e31791c5109 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:09.824209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c066c-4385-4c91-9d0f-9e31791c5109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:09.844974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-1a7c066c-4385-4c91-9d0f-9e31791c5109 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:09.846214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1a7c066c-4385-4c91-9d0f-9e31791c5109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:09.854136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 980/1957] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:09.858535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3b2a7-79d6-4fef-863c-ca3b494339a7 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:09.868989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3b2a7-79d6-4fef-863c-ca3b494339a7 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:09.873571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f873df-2b34-430b-be00-1b3e1736141f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:09.874174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f873df-2b34-430b-be00-1b3e1736141f admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:09.874450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f873df-2b34-430b-be00-1b3e1736141f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:09.874720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f873df-2b34-430b-be00-1b3e1736141f admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:09.875367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f873df-2b34-430b-be00-1b3e1736141f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:53:09.875669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f873df-2b34-430b-be00-1b3e1736141f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:09.877920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f873df-2b34-430b-be00-1b3e1736141f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:09.889391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 978/1958] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:53:09.893788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c066c-4385-4c91-9d0f-9e31791c5109 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:09.902856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c066c-4385-4c91-9d0f-9e31791c5109 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:09.907465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231815d7-f5f6-4673-9709-942d3a9ec4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:09.908077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231815d7-f5f6-4673-9709-942d3a9ec4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:09.908358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231815d7-f5f6-4673-9709-942d3a9ec4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:09.908627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231815d7-f5f6-4673-9709-942d3a9ec4b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:09.909264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231815d7-f5f6-4673-9709-942d3a9ec4b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:09.911194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231815d7-f5f6-4673-9709-942d3a9ec4b6 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:09.940787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 981/1959] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:09 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:09.945206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f873df-2b34-430b-be00-1b3e1736141f admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:09.955392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f873df-2b34-430b-be00-1b3e1736141f admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:10.012512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a842a1-fa36-4a2a-9abc-af39506e73ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:10.013110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a842a1-fa36-4a2a-9abc-af39506e73ca admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:10.013384 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a842a1-fa36-4a2a-9abc-af39506e73ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:10.013678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a842a1-fa36-4a2a-9abc-af39506e73ca admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:10.014310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a842a1-fa36-4a2a-9abc-af39506e73ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:10.016199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a842a1-fa36-4a2a-9abc-af39506e73ca admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:10.038387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 979/1960] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:09 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:10.043060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231815d7-f5f6-4673-9709-942d3a9ec4b6 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:10.058297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231815d7-f5f6-4673-9709-942d3a9ec4b6 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:10.063219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3eb287-813b-4e28-ac4e-32b19619f994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:10.063943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3eb287-813b-4e28-ac4e-32b19619f994 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:10.064284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3eb287-813b-4e28-ac4e-32b19619f994 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:10.064629 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3eb287-813b-4e28-ac4e-32b19619f994 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:10.065380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3eb287-813b-4e28-ac4e-32b19619f994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:10.067042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3eb287-813b-4e28-ac4e-32b19619f994 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:10.077646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 982/1961] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:10.082372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a842a1-fa36-4a2a-9abc-af39506e73ca admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:10.094324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a842a1-fa36-4a2a-9abc-af39506e73ca admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:10.099949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefd4fd-b6a8-4b0f-bcfb-d50677cb4c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:10.100505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefd4fd-b6a8-4b0f-bcfb-d50677cb4c8a admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:10.100798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefd4fd-b6a8-4b0f-bcfb-d50677cb4c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:10.101077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefd4fd-b6a8-4b0f-bcfb-d50677cb4c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:10.101914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefd4fd-b6a8-4b0f-bcfb-d50677cb4c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:10.103702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefd4fd-b6a8-4b0f-bcfb-d50677cb4c8a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:10.115670 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 980/1962] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:10.121666 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3eb287-813b-4e28-ac4e-32b19619f994 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:10.132343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3eb287-813b-4e28-ac4e-32b19619f994 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:10.136261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616cf03-f36a-4f93-9862-6145ccbbc5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:10.137009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616cf03-f36a-4f93-9862-6145ccbbc5f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:10.137165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616cf03-f36a-4f93-9862-6145ccbbc5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:10.137443 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616cf03-f36a-4f93-9862-6145ccbbc5f5 admin admin] PATH_INFO: `/v3/projects/91db21eb8a854959b6ac88517b6da9ae/users/5097e22414e04ef88a62d261f66e8d08/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:10.153175 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616cf03-f36a-4f93-9862-6145ccbbc5f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91db21eb8a854959b6ac88517b6da9ae, user_id=5097e22414e04ef88a62d261f66e8d08, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:10.155993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616cf03-f36a-4f93-9862-6145ccbbc5f5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:10.176261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 983/1963] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:10 2022] PUT /identity/v3/projects/91db21eb8a854959b6ac88517b6da9ae/users/5097e22414e04ef88a62d261f66e8d08/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:10.186337 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e890c-23b4-417c-9f87-ae74e3107205 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:10.186622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e890c-23b4-417c-9f87-ae74e3107205 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:10.187073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e890c-23b4-417c-9f87-ae74e3107205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:10.215184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ac8e890c-23b4-417c-9f87-ae74e3107205 None None] MFA Rules not processed for user `5097e22414e04ef88a62d261f66e8d08`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:10.216335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ac8e890c-23b4-417c-9f87-ae74e3107205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:10.286341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:10.301999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 981/1964] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:10.310312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616cf03-f36a-4f93-9862-6145ccbbc5f5 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:10.312476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d616cf03-f36a-4f93-9862-6145ccbbc5f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:10.367411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:10.380125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616cf03-f36a-4f93-9862-6145ccbbc5f5 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:10.383980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e32da7-2454-420d-8299-6877cbe12fd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:10.384592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e32da7-2454-420d-8299-6877cbe12fd7 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:10.384874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e32da7-2454-420d-8299-6877cbe12fd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:10.385151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e32da7-2454-420d-8299-6877cbe12fd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:10.394287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e32da7-2454-420d-8299-6877cbe12fd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:10.395868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e32da7-2454-420d-8299-6877cbe12fd7 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:10.404842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 984/1965] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:10.428621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e890c-23b4-417c-9f87-ae74e3107205 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:10.502022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e890c-23b4-417c-9f87-ae74e3107205 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:10.507882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:10.509201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:10.509589 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:10.509944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] PATH_INFO: `/v3/users/5097e22414e04ef88a62d261f66e8d08` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:10.511338 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5097e22414e04ef88a62d261f66e8d08)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:10.514019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:10.583233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5097e22414e04ef88a62d261f66e8d08'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:10.584225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5097e22414e04ef88a62d261f66e8d08'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:10.592968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5097e22414e04ef88a62d261f66e8d08'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:10.604295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5097e22414e04ef88a62d261f66e8d08'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:10.605208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5097e22414e04ef88a62d261f66e8d08'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:10.625552 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 982/1966] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:10 2022] DELETE /identity/v3/users/5097e22414e04ef88a62d261f66e8d08 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:10.631242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6382429-df51-4e9f-8dc3-88d9b96c7e9d None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:10.631562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6382429-df51-4e9f-8dc3-88d9b96c7e9d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:10.631842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6382429-df51-4e9f-8dc3-88d9b96c7e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:10.651956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e6382429-df51-4e9f-8dc3-88d9b96c7e9d None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:10.653179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e6382429-df51-4e9f-8dc3-88d9b96c7e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:10.715720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 985/1967] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:10.725751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:10.727198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:10.783121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb005cf-35a3-4e1f-af2d-4b07dc1cb6b1 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:10.786661 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a7506-55e7-45ab-b9a8-a74c04502040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:10.787252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a7506-55e7-45ab-b9a8-a74c04502040 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:10.787527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a7506-55e7-45ab-b9a8-a74c04502040 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:10.787791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a7506-55e7-45ab-b9a8-a74c04502040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:10.796454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a7506-55e7-45ab-b9a8-a74c04502040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:10.797913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a7506-55e7-45ab-b9a8-a74c04502040 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:10.807611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 983/1968] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:53:11.073307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6382429-df51-4e9f-8dc3-88d9b96c7e9d None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:11.074915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e6382429-df51-4e9f-8dc3-88d9b96c7e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:11.087173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6382429-df51-4e9f-8dc3-88d9b96c7e9d None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:11.091785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:11.092709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:11.092993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:11.093268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] PATH_INFO: `/v3/projects/91db21eb8a854959b6ac88517b6da9ae` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:11.094457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] RBAC: Authorizing `identity:delete_project(project_id=91db21eb8a854959b6ac88517b6da9ae)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:11.096924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:11.145969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91db21eb8a854959b6ac88517b6da9ae', 'request_id': 'req-0b5e6390-587d-43de-b4a5-2561b83d420e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:11.153933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91db21eb8a854959b6ac88517b6da9ae', 'request_id': 'req-0b5e6390-587d-43de-b4a5-2561b83d420e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:11.162179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91db21eb8a854959b6ac88517b6da9ae', 'request_id': 'req-0b5e6390-587d-43de-b4a5-2561b83d420e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:11.177691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] The token cache is being invalidate because project 91db21eb8a854959b6ac88517b6da9ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:11.178050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:11.178822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0b5e6390-587d-43de-b4a5-2561b83d420e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:11.180582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 986/1969] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:11 2022] DELETE /identity/v3/projects/91db21eb8a854959b6ac88517b6da9ae => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:11.319061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a7506-55e7-45ab-b9a8-a74c04502040 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:11.320520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8a0a7506-55e7-45ab-b9a8-a74c04502040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:11.333819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a7506-55e7-45ab-b9a8-a74c04502040 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:11.350480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:11.400736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:11.401522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:11.401922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:11.402290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] PATH_INFO: `/v3/users/6ec2e8bd8f184e6fa954916cd25e203d` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:11.403496 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec2e8bd8f184e6fa954916cd25e203d)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:11.405558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:11.452900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec2e8bd8f184e6fa954916cd25e203d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:11.453601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec2e8bd8f184e6fa954916cd25e203d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:11.461386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec2e8bd8f184e6fa954916cd25e203d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:11.469239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec2e8bd8f184e6fa954916cd25e203d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:11.469932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec2e8bd8f184e6fa954916cd25e203d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:11.488612 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 984/1970] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:11 2022] DELETE /identity/v3/users/6ec2e8bd8f184e6fa954916cd25e203d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:11.493103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b94b60-275f-4e94-8e86-984483e2d1c3 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:11.493394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b94b60-275f-4e94-8e86-984483e2d1c3 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:11.493695 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b94b60-275f-4e94-8e86-984483e2d1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:11.512930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d9b94b60-275f-4e94-8e86-984483e2d1c3 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:11.514304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d9b94b60-275f-4e94-8e86-984483e2d1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:11.575782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 987/1971] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:11.586012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:11.587388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:11.644425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dea32b-7755-404f-928b-ef62f163eae3 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:11.648129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d7b66-9876-49f5-b0d2-8ccd63ff38b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:11.648731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d7b66-9876-49f5-b0d2-8ccd63ff38b6 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:11.649017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d7b66-9876-49f5-b0d2-8ccd63ff38b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:11.649289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d7b66-9876-49f5-b0d2-8ccd63ff38b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:11.658946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d7b66-9876-49f5-b0d2-8ccd63ff38b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:11.660379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d7b66-9876-49f5-b0d2-8ccd63ff38b6 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:11.669841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 985/1972] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:53:11.996391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b94b60-275f-4e94-8e86-984483e2d1c3 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:11.998041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d9b94b60-275f-4e94-8e86-984483e2d1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:12.012148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b94b60-275f-4e94-8e86-984483e2d1c3 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:12.016641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:12.017959 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:12.018142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:12.018410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] PATH_INFO: `/v3/projects/4d1bb3f5687f47c18d4a75601d92edd6` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:12.019521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1bb3f5687f47c18d4a75601d92edd6)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:12.021585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:12.028285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:12.073774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1bb3f5687f47c18d4a75601d92edd6', 'request_id': 'req-9046a152-5f20-439f-81b3-555e9a432308'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.081706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1bb3f5687f47c18d4a75601d92edd6', 'request_id': 'req-9046a152-5f20-439f-81b3-555e9a432308'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.087796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1bb3f5687f47c18d4a75601d92edd6', 'request_id': 'req-9046a152-5f20-439f-81b3-555e9a432308'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.098176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] The token cache is being invalidate because project 4d1bb3f5687f47c18d4a75601d92edd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:12.098462 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.099089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.101032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 988/1973] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:11 2022] DELETE /identity/v3/projects/4d1bb3f5687f47c18d4a75601d92edd6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:12.105255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d7b66-9876-49f5-b0d2-8ccd63ff38b6 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:12.115581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d7b66-9876-49f5-b0d2-8ccd63ff38b6 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:12.152593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:12.172163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:12.172786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:12.173640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:12.173937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] PATH_INFO: `/v3/users/7270a704f73f4e4a837925d669c903a0` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:12.175174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7270a704f73f4e4a837925d669c903a0)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:12.177118 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:12.222937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7270a704f73f4e4a837925d669c903a0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.223582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7270a704f73f4e4a837925d669c903a0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.230613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7270a704f73f4e4a837925d669c903a0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.238746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7270a704f73f4e4a837925d669c903a0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.239430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-55a14786-0f4a-4094-b2f5-4c88659a94dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7270a704f73f4e4a837925d669c903a0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.260237 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 986/1974] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:12 2022] DELETE /identity/v3/users/7270a704f73f4e4a837925d669c903a0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:12.528736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:12.530398 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:12.544474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046a152-5f20-439f-81b3-555e9a432308 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:12.605242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:12.605910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:12.606188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:12.606457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] PATH_INFO: `/v3/projects/b69a73d905f9451288dbba24ea46725e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:12.607575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69a73d905f9451288dbba24ea46725e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:12.609478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:12.659403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69a73d905f9451288dbba24ea46725e', 'request_id': 'req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.666513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69a73d905f9451288dbba24ea46725e', 'request_id': 'req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.673965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69a73d905f9451288dbba24ea46725e', 'request_id': 'req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.684518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] The token cache is being invalidate because project b69a73d905f9451288dbba24ea46725e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:12.684805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.685446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7ef2d25f-14e4-4a7c-9dc4-c5478d90009e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:12.686973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 989/1975] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:12 2022] DELETE /identity/v3/projects/b69a73d905f9451288dbba24ea46725e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:12.691986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139aa9c-2f81-4d27-bad0-5097ec2e94db None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:12.692381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139aa9c-2f81-4d27-bad0-5097ec2e94db None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:12.692525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139aa9c-2f81-4d27-bad0-5097ec2e94db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:12.711804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b139aa9c-2f81-4d27-bad0-5097ec2e94db None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:12.712993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b139aa9c-2f81-4d27-bad0-5097ec2e94db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:12.769925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 987/1976] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:12.777809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09934f77-55e6-4eff-81a6-d1e312a1af77 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:12.778038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09934f77-55e6-4eff-81a6-d1e312a1af77 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:12.778310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09934f77-55e6-4eff-81a6-d1e312a1af77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:12.798856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-09934f77-55e6-4eff-81a6-d1e312a1af77 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:12.800055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-09934f77-55e6-4eff-81a6-d1e312a1af77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:12.807893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 990/1977] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:12.811976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139aa9c-2f81-4d27-bad0-5097ec2e94db None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:12.821414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139aa9c-2f81-4d27-bad0-5097ec2e94db None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:12.825937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b98ff-725e-47df-9107-e74b872613ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:12.826542 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b98ff-725e-47df-9107-e74b872613ca admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:12.826844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b98ff-725e-47df-9107-e74b872613ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:12.827089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b98ff-725e-47df-9107-e74b872613ca admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:12.827738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b98ff-725e-47df-9107-e74b872613ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:53:12.828044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b98ff-725e-47df-9107-e74b872613ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:12.829673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b98ff-725e-47df-9107-e74b872613ca admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:12.845608 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 988/1978] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:53:12.846666 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09934f77-55e6-4eff-81a6-d1e312a1af77 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:12.855942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09934f77-55e6-4eff-81a6-d1e312a1af77 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:12.861217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6278e0-a555-49e1-ae94-9f7dbc1989db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:12.861943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6278e0-a555-49e1-ae94-9f7dbc1989db admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:12.862222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6278e0-a555-49e1-ae94-9f7dbc1989db admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:12.862493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6278e0-a555-49e1-ae94-9f7dbc1989db admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:12.863131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6278e0-a555-49e1-ae94-9f7dbc1989db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:12.865307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6278e0-a555-49e1-ae94-9f7dbc1989db admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:12.881480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:12.900011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 991/1979] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:12 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:12.904474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b98ff-725e-47df-9107-e74b872613ca admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:12.916078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b98ff-725e-47df-9107-e74b872613ca admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:12.961922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:12.977759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e71e8a-e4c7-42aa-99bc-33ff3ed54f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:12.978373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e71e8a-e4c7-42aa-99bc-33ff3ed54f30 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:12.978685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e71e8a-e4c7-42aa-99bc-33ff3ed54f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:12.978961 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e71e8a-e4c7-42aa-99bc-33ff3ed54f30 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:12.979605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e71e8a-e4c7-42aa-99bc-33ff3ed54f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:12.982162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e71e8a-e4c7-42aa-99bc-33ff3ed54f30 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.006626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 989/1980] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:12 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:13.007795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6278e0-a555-49e1-ae94-9f7dbc1989db admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:13.018485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6278e0-a555-49e1-ae94-9f7dbc1989db admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:13.023085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a5350-fd83-4706-9f8f-b3098fae7f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:13.023690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a5350-fd83-4706-9f8f-b3098fae7f42 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.023971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a5350-fd83-4706-9f8f-b3098fae7f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.024243 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a5350-fd83-4706-9f8f-b3098fae7f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.024903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a5350-fd83-4706-9f8f-b3098fae7f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:13.027281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a5350-fd83-4706-9f8f-b3098fae7f42 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.038173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 992/1981] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:53:13.042148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e71e8a-e4c7-42aa-99bc-33ff3ed54f30 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:13.052042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e71e8a-e4c7-42aa-99bc-33ff3ed54f30 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:13.056536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1189b08-f639-494b-a331-56c723f8ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:13.057154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1189b08-f639-494b-a331-56c723f8ffe2 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.057413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1189b08-f639-494b-a331-56c723f8ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.057711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1189b08-f639-494b-a331-56c723f8ffe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.058351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1189b08-f639-494b-a331-56c723f8ffe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:13.059928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1189b08-f639-494b-a331-56c723f8ffe2 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.070791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 990/1982] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:13.075329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a5350-fd83-4706-9f8f-b3098fae7f42 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:13.086460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a5350-fd83-4706-9f8f-b3098fae7f42 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:13.091413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3dec7-a690-4c51-ad4f-d7df69d1a892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:13.092076 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3dec7-a690-4c51-ad4f-d7df69d1a892 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.092610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3dec7-a690-4c51-ad4f-d7df69d1a892 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.092887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3dec7-a690-4c51-ad4f-d7df69d1a892 admin admin] PATH_INFO: `/v3/projects/91dd7bc2f4394a4cb192fc4d04145d6d/users/a394fbee734a412f8257f63e01ece9cd/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.108291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3dec7-a690-4c51-ad4f-d7df69d1a892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91dd7bc2f4394a4cb192fc4d04145d6d, user_id=a394fbee734a412f8257f63e01ece9cd, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:13.111024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3dec7-a690-4c51-ad4f-d7df69d1a892 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.131776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 993/1983] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:13 2022] PUT /identity/v3/projects/91dd7bc2f4394a4cb192fc4d04145d6d/users/a394fbee734a412f8257f63e01ece9cd/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:13.144616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd291dfb-af8c-47fd-b953-e5c56626da07 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.144944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd291dfb-af8c-47fd-b953-e5c56626da07 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.145159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd291dfb-af8c-47fd-b953-e5c56626da07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.176235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-cd291dfb-af8c-47fd-b953-e5c56626da07 None None] MFA Rules not processed for user `a394fbee734a412f8257f63e01ece9cd`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:13.177676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-cd291dfb-af8c-47fd-b953-e5c56626da07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:13.273910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 991/1984] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:13.281650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3dec7-a690-4c51-ad4f-d7df69d1a892 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:13.355655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3dec7-a690-4c51-ad4f-d7df69d1a892 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:13.360673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a06ed6-185b-4abb-82d4-c3c77ee4c4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:13.361283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a06ed6-185b-4abb-82d4-c3c77ee4c4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.361588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a06ed6-185b-4abb-82d4-c3c77ee4c4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.361863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a06ed6-185b-4abb-82d4-c3c77ee4c4c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.362497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a06ed6-185b-4abb-82d4-c3c77ee4c4c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:13.364416 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a06ed6-185b-4abb-82d4-c3c77ee4c4c0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.391236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 994/1985] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:13 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:13.395762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd291dfb-af8c-47fd-b953-e5c56626da07 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:13.406424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd291dfb-af8c-47fd-b953-e5c56626da07 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:13.461599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927686c5-18f6-4628-aa7f-080c211cbd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:13.462214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927686c5-18f6-4628-aa7f-080c211cbd00 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.462495 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927686c5-18f6-4628-aa7f-080c211cbd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.462775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927686c5-18f6-4628-aa7f-080c211cbd00 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.463425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927686c5-18f6-4628-aa7f-080c211cbd00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:13.465340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927686c5-18f6-4628-aa7f-080c211cbd00 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.486581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 992/1986] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:13 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:13.490711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a06ed6-185b-4abb-82d4-c3c77ee4c4c0 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:13.500954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a06ed6-185b-4abb-82d4-c3c77ee4c4c0 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:13.506087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072336f-d881-40f5-9ca9-14d65ee021ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:13.506684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072336f-d881-40f5-9ca9-14d65ee021ad admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.506967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072336f-d881-40f5-9ca9-14d65ee021ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.507243 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072336f-d881-40f5-9ca9-14d65ee021ad admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.507892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072336f-d881-40f5-9ca9-14d65ee021ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:13.509508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072336f-d881-40f5-9ca9-14d65ee021ad admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.520454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 995/1987] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:13.524428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927686c5-18f6-4628-aa7f-080c211cbd00 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:13.534252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927686c5-18f6-4628-aa7f-080c211cbd00 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:13.539052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf04863-affa-4551-be64-26ce2650ed63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:13.539807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf04863-affa-4551-be64-26ce2650ed63 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.540089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf04863-affa-4551-be64-26ce2650ed63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.540371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf04863-affa-4551-be64-26ce2650ed63 admin admin] PATH_INFO: `/v3/projects/a48ebc10a3154fe4a6330c21b4235161/users/f09288508760470881cbc35bb2ba3056/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.555383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf04863-affa-4551-be64-26ce2650ed63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48ebc10a3154fe4a6330c21b4235161, user_id=f09288508760470881cbc35bb2ba3056, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:13.558076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf04863-affa-4551-be64-26ce2650ed63 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.571938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 993/1988] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:13 2022] PUT /identity/v3/projects/a48ebc10a3154fe4a6330c21b4235161/users/f09288508760470881cbc35bb2ba3056/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:53:13.575908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072336f-d881-40f5-9ca9-14d65ee021ad admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:13.586374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072336f-d881-40f5-9ca9-14d65ee021ad admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:13.641123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053e21b-a6e2-45c7-95ec-e1675347ed45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:13.641755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053e21b-a6e2-45c7-95ec-e1675347ed45 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.642032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053e21b-a6e2-45c7-95ec-e1675347ed45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.642300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053e21b-a6e2-45c7-95ec-e1675347ed45 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.642957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053e21b-a6e2-45c7-95ec-e1675347ed45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:13.644540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053e21b-a6e2-45c7-95ec-e1675347ed45 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.653960 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 996/1989] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:13.658068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf04863-affa-4551-be64-26ce2650ed63 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:13.668281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf04863-affa-4551-be64-26ce2650ed63 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:13.672788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b79c4-e205-4652-9a5e-7a8860be7175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:13.674301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b79c4-e205-4652-9a5e-7a8860be7175 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.674582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b79c4-e205-4652-9a5e-7a8860be7175 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.674850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b79c4-e205-4652-9a5e-7a8860be7175 admin admin] PATH_INFO: `/v3/projects/a48ebc10a3154fe4a6330c21b4235161/users/f09288508760470881cbc35bb2ba3056/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.676743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b79c4-e205-4652-9a5e-7a8860be7175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48ebc10a3154fe4a6330c21b4235161, user_id=f09288508760470881cbc35bb2ba3056, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:13.679419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b79c4-e205-4652-9a5e-7a8860be7175 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.690331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 994/1990] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:13 2022] PUT /identity/v3/projects/a48ebc10a3154fe4a6330c21b4235161/users/f09288508760470881cbc35bb2ba3056/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:13.700431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec36dd-9fc9-435c-91f2-547d773bbebe None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.700723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec36dd-9fc9-435c-91f2-547d773bbebe None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.701043 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec36dd-9fc9-435c-91f2-547d773bbebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.730268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d5ec36dd-9fc9-435c-91f2-547d773bbebe None None] MFA Rules not processed for user `f09288508760470881cbc35bb2ba3056`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:13.731475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d5ec36dd-9fc9-435c-91f2-547d773bbebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:13.750358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:13.802407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 997/1991] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:13.814565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b79c4-e205-4652-9a5e-7a8860be7175 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:13.816283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-df0b79c4-e205-4652-9a5e-7a8860be7175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:13.826167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:13.889803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b79c4-e205-4652-9a5e-7a8860be7175 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:13.893902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb10ed-ff85-4225-b3b1-3fa3b3a16b19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:13.895417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb10ed-ff85-4225-b3b1-3fa3b3a16b19 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:13.895792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb10ed-ff85-4225-b3b1-3fa3b3a16b19 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:13.896165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb10ed-ff85-4225-b3b1-3fa3b3a16b19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:13.910869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb10ed-ff85-4225-b3b1-3fa3b3a16b19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:13.912479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb10ed-ff85-4225-b3b1-3fa3b3a16b19 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:13.927663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 995/1992] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:14.010879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec36dd-9fc9-435c-91f2-547d773bbebe None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:14.044675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec36dd-9fc9-435c-91f2-547d773bbebe None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:14.052275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88db39f-a0fd-405d-8a37-3350c24ba1ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:14.052974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88db39f-a0fd-405d-8a37-3350c24ba1ef service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:14.053177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88db39f-a0fd-405d-8a37-3350c24ba1ef service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:14.053446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88db39f-a0fd-405d-8a37-3350c24ba1ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:14.131734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88db39f-a0fd-405d-8a37-3350c24ba1ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:14.133673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88db39f-a0fd-405d-8a37-3350c24ba1ef service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:14.145069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 998/1993] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:23.919303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=23 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:24.139557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=48 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:26.772555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb10ed-ff85-4225-b3b1-3fa3b3a16b19 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:26.846861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb10ed-ff85-4225-b3b1-3fa3b3a16b19 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:26.851545 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:26.852180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:26.852457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:26.852725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] PATH_INFO: `/v3/users/a394fbee734a412f8257f63e01ece9cd` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:26.853865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a394fbee734a412f8257f63e01ece9cd)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:26.855772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:26.915112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a394fbee734a412f8257f63e01ece9cd'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:26.917884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a394fbee734a412f8257f63e01ece9cd'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:26.926595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a394fbee734a412f8257f63e01ece9cd'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:26.936293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a394fbee734a412f8257f63e01ece9cd'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:26.936995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a394fbee734a412f8257f63e01ece9cd'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:26.956354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 996/1994] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:26 2022] DELETE /identity/v3/users/a394fbee734a412f8257f63e01ece9cd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:26.960844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91670a3-1b0f-418f-b49c-707cea9d88a5 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:26.961345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91670a3-1b0f-418f-b49c-707cea9d88a5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:26.961635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91670a3-1b0f-418f-b49c-707cea9d88a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:26.982192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e91670a3-1b0f-418f-b49c-707cea9d88a5 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:26.983533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e91670a3-1b0f-418f-b49c-707cea9d88a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:27.053076 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 999/1995] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:27.062647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:27.064055 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:27.120236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f6557-53c4-4a5f-aa85-645deb700031 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:27.124128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4d475-9a64-4988-8941-e1c4d184aad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:27.124748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4d475-9a64-4988-8941-e1c4d184aad7 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:27.125030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4d475-9a64-4988-8941-e1c4d184aad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:27.125432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4d475-9a64-4988-8941-e1c4d184aad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:27.135043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4d475-9a64-4988-8941-e1c4d184aad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:27.136517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4d475-9a64-4988-8941-e1c4d184aad7 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:27.146705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 997/1996] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:27.481682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91670a3-1b0f-418f-b49c-707cea9d88a5 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:27.483188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e91670a3-1b0f-418f-b49c-707cea9d88a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:27.495447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91670a3-1b0f-418f-b49c-707cea9d88a5 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:27.500989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:27.501852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:27.502136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:27.502410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] PATH_INFO: `/v3/projects/91dd7bc2f4394a4cb192fc4d04145d6d` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:27.503564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91dd7bc2f4394a4cb192fc4d04145d6d)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:27.505486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:27.552619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91dd7bc2f4394a4cb192fc4d04145d6d', 'request_id': 'req-c7815824-3b3b-4ca3-9f82-a8f749f99466'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:27.558997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91dd7bc2f4394a4cb192fc4d04145d6d', 'request_id': 'req-c7815824-3b3b-4ca3-9f82-a8f749f99466'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:27.565817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91dd7bc2f4394a4cb192fc4d04145d6d', 'request_id': 'req-c7815824-3b3b-4ca3-9f82-a8f749f99466'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:27.575886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] The token cache is being invalidate because project 91dd7bc2f4394a4cb192fc4d04145d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:27.576197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:27.576825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:27.579194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1000/1997] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:27 2022] DELETE /identity/v3/projects/91dd7bc2f4394a4cb192fc4d04145d6d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:27.583330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4d475-9a64-4988-8941-e1c4d184aad7 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:27.593439 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4d475-9a64-4988-8941-e1c4d184aad7 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:27.611743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:27.651052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:27.651611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:27.651891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:27.652163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] PATH_INFO: `/v3/users/f09288508760470881cbc35bb2ba3056` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:27.653298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f09288508760470881cbc35bb2ba3056)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:27.655288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:27.700878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f09288508760470881cbc35bb2ba3056'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:27.701519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f09288508760470881cbc35bb2ba3056'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:27.709003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f09288508760470881cbc35bb2ba3056'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:27.716355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f09288508760470881cbc35bb2ba3056'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:27.717254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-044f14e4-6490-439e-b597-d661add78143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f09288508760470881cbc35bb2ba3056'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:27.734349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 998/1998] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:27 2022] DELETE /identity/v3/users/f09288508760470881cbc35bb2ba3056 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:53:28.011153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:28.012795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:28.034522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7815824-3b3b-4ca3-9f82-a8f749f99466 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:28.100446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:28.101091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.101757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.101883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] PATH_INFO: `/v3/projects/a48ebc10a3154fe4a6330c21b4235161` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.103186 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a48ebc10a3154fe4a6330c21b4235161)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:28.105236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:28.145827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:28.171503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a48ebc10a3154fe4a6330c21b4235161', 'request_id': 'req-b67bf0be-1108-47d9-9092-fdfc9f450d26'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:28.181788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a48ebc10a3154fe4a6330c21b4235161', 'request_id': 'req-b67bf0be-1108-47d9-9092-fdfc9f450d26'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:28.191360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a48ebc10a3154fe4a6330c21b4235161', 'request_id': 'req-b67bf0be-1108-47d9-9092-fdfc9f450d26'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:28.204331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] The token cache is being invalidate because project a48ebc10a3154fe4a6330c21b4235161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:28.204922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:28.205639 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b67bf0be-1108-47d9-9092-fdfc9f450d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:28.210649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1001/1999] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:28 2022] DELETE /identity/v3/projects/a48ebc10a3154fe4a6330c21b4235161 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:28.212691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1517a-260e-465c-b41e-1d666e3d9680 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.213328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1517a-260e-465c-b41e-1d666e3d9680 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.213653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1517a-260e-465c-b41e-1d666e3d9680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.242665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-fde1517a-260e-465c-b41e-1d666e3d9680 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:28.244591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-fde1517a-260e-465c-b41e-1d666e3d9680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:28.327204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 999/2000] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:28.334995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1ad2d-5764-4950-a3a1-a0ec4fe7d65c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.335286 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1ad2d-5764-4950-a3a1-a0ec4fe7d65c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.335565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1ad2d-5764-4950-a3a1-a0ec4fe7d65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.359161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6cb1ad2d-5764-4950-a3a1-a0ec4fe7d65c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:28.361215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6cb1ad2d-5764-4950-a3a1-a0ec4fe7d65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:28.369307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1002/2001] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:28.373692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1517a-260e-465c-b41e-1d666e3d9680 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:28.384185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1517a-260e-465c-b41e-1d666e3d9680 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:28.388746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386f137-2ad6-45d7-b5f1-c70f2dbcab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:28.389344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386f137-2ad6-45d7-b5f1-c70f2dbcab41 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.389655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386f137-2ad6-45d7-b5f1-c70f2dbcab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.389924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386f137-2ad6-45d7-b5f1-c70f2dbcab41 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.390573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386f137-2ad6-45d7-b5f1-c70f2dbcab41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:53:28.390898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386f137-2ad6-45d7-b5f1-c70f2dbcab41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:28.392500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386f137-2ad6-45d7-b5f1-c70f2dbcab41 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:28.404383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1000/2002] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:53:28.409261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1ad2d-5764-4950-a3a1-a0ec4fe7d65c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:28.418431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1ad2d-5764-4950-a3a1-a0ec4fe7d65c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:28.423701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a4dbe-aacc-48dc-b290-4da6456f9a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:28.424299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a4dbe-aacc-48dc-b290-4da6456f9a18 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.424583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a4dbe-aacc-48dc-b290-4da6456f9a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.424860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a4dbe-aacc-48dc-b290-4da6456f9a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.425512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a4dbe-aacc-48dc-b290-4da6456f9a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:28.427774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a4dbe-aacc-48dc-b290-4da6456f9a18 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:28.454185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1003/2003] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:28 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:28.458146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386f137-2ad6-45d7-b5f1-c70f2dbcab41 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:28.468213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386f137-2ad6-45d7-b5f1-c70f2dbcab41 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:28.494879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:28.525839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72341590-93ea-497c-ac28-d251ccd694ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:28.526440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72341590-93ea-497c-ac28-d251ccd694ec admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.526713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72341590-93ea-497c-ac28-d251ccd694ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.526979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72341590-93ea-497c-ac28-d251ccd694ec admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.527658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72341590-93ea-497c-ac28-d251ccd694ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:28.529581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72341590-93ea-497c-ac28-d251ccd694ec admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:28.551773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1001/2004] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:28 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:28.555937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a4dbe-aacc-48dc-b290-4da6456f9a18 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:28.568905 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a4dbe-aacc-48dc-b290-4da6456f9a18 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:28.573655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9f86d-dbc1-425c-8e4b-7f5a1a0e7ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:28.574266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9f86d-dbc1-425c-8e4b-7f5a1a0e7ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.574585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9f86d-dbc1-425c-8e4b-7f5a1a0e7ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.574957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9f86d-dbc1-425c-8e4b-7f5a1a0e7ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.575620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9f86d-dbc1-425c-8e4b-7f5a1a0e7ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:28.577252 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9f86d-dbc1-425c-8e4b-7f5a1a0e7ca8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:28.587778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1004/2005] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:28.591916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72341590-93ea-497c-ac28-d251ccd694ec admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:28.602844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72341590-93ea-497c-ac28-d251ccd694ec admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:28.607495 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955fe55-a156-4d01-b53b-be640bcb3d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:28.608102 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955fe55-a156-4d01-b53b-be640bcb3d8b admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.608387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955fe55-a156-4d01-b53b-be640bcb3d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.608659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955fe55-a156-4d01-b53b-be640bcb3d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.609314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955fe55-a156-4d01-b53b-be640bcb3d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:28.611548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955fe55-a156-4d01-b53b-be640bcb3d8b admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:28.621886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1002/2006] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:28.625933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9f86d-dbc1-425c-8e4b-7f5a1a0e7ca8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:28.635914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9f86d-dbc1-425c-8e4b-7f5a1a0e7ca8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:28.640634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa1f3f-4ef0-4ed7-8454-c6a9a77b60d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:28.641816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa1f3f-4ef0-4ed7-8454-c6a9a77b60d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.642141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa1f3f-4ef0-4ed7-8454-c6a9a77b60d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.642417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa1f3f-4ef0-4ed7-8454-c6a9a77b60d3 admin admin] PATH_INFO: `/v3/projects/ea045c28d6c848b9a49a73347c9b9e33/users/78fdacaa8cc14d31a9d98cb543ab305c/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.657744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa1f3f-4ef0-4ed7-8454-c6a9a77b60d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea045c28d6c848b9a49a73347c9b9e33, user_id=78fdacaa8cc14d31a9d98cb543ab305c, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:28.660402 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa1f3f-4ef0-4ed7-8454-c6a9a77b60d3 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:28.675506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1005/2007] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:28 2022] PUT /identity/v3/projects/ea045c28d6c848b9a49a73347c9b9e33/users/78fdacaa8cc14d31a9d98cb543ab305c/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:28.687299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177b2a3-0735-4d18-a616-5a67427fead5 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.687596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177b2a3-0735-4d18-a616-5a67427fead5 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.687899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177b2a3-0735-4d18-a616-5a67427fead5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.716545 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5177b2a3-0735-4d18-a616-5a67427fead5 None None] MFA Rules not processed for user `78fdacaa8cc14d31a9d98cb543ab305c`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:28.717981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5177b2a3-0735-4d18-a616-5a67427fead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:28.781165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1003/2008] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:28.786451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa1f3f-4ef0-4ed7-8454-c6a9a77b60d3 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:28.846554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa1f3f-4ef0-4ed7-8454-c6a9a77b60d3 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:28.851157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17c3fd-6a06-4318-87b7-5666f018b783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:28.851751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17c3fd-6a06-4318-87b7-5666f018b783 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.852032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17c3fd-6a06-4318-87b7-5666f018b783 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.852302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17c3fd-6a06-4318-87b7-5666f018b783 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.852937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17c3fd-6a06-4318-87b7-5666f018b783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:28.854875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17c3fd-6a06-4318-87b7-5666f018b783 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:28.879948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1006/2009] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:28 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:28.884324 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177b2a3-0735-4d18-a616-5a67427fead5 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:28.895495 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177b2a3-0735-4d18-a616-5a67427fead5 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:28.951128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd60447-6934-4ed0-a385-e89472fab9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:28.951723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd60447-6934-4ed0-a385-e89472fab9af admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.951998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd60447-6934-4ed0-a385-e89472fab9af admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.952263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd60447-6934-4ed0-a385-e89472fab9af admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.952894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd60447-6934-4ed0-a385-e89472fab9af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:28.955341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd60447-6934-4ed0-a385-e89472fab9af admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:28.976342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1004/2010] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:28 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:28.980497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17c3fd-6a06-4318-87b7-5666f018b783 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:28.990827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17c3fd-6a06-4318-87b7-5666f018b783 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:28.995450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3c5fd-54de-4e12-8bd1-06f627e88b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:28.996046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3c5fd-54de-4e12-8bd1-06f627e88b0b admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:28.996331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3c5fd-54de-4e12-8bd1-06f627e88b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:28.996608 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3c5fd-54de-4e12-8bd1-06f627e88b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:28.997293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3c5fd-54de-4e12-8bd1-06f627e88b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:28.998956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3c5fd-54de-4e12-8bd1-06f627e88b0b admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:29.009878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1007/2011] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:29.013599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd60447-6934-4ed0-a385-e89472fab9af admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:29.023881 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd60447-6934-4ed0-a385-e89472fab9af admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:29.028368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94220a5a-13b4-43d9-bf32-b61734f83b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:29.028949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94220a5a-13b4-43d9-bf32-b61734f83b34 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.029223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94220a5a-13b4-43d9-bf32-b61734f83b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.029488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94220a5a-13b4-43d9-bf32-b61734f83b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:29.030152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94220a5a-13b4-43d9-bf32-b61734f83b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:29.031771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94220a5a-13b4-43d9-bf32-b61734f83b34 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:29.041670 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1005/2012] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:29.045711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3c5fd-54de-4e12-8bd1-06f627e88b0b admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:29.055498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3c5fd-54de-4e12-8bd1-06f627e88b0b admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:29.060098 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5ee42-f2de-40f1-a161-f2d9275b0475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:29.060738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5ee42-f2de-40f1-a161-f2d9275b0475 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.061037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5ee42-f2de-40f1-a161-f2d9275b0475 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.061313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5ee42-f2de-40f1-a161-f2d9275b0475 admin admin] PATH_INFO: `/v3/projects/41584b9207a2459ebd0d0a8cab7c4a1b/users/d1d99a3a66014dabaa993acc467a9672/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:29.075801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5ee42-f2de-40f1-a161-f2d9275b0475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41584b9207a2459ebd0d0a8cab7c4a1b, user_id=d1d99a3a66014dabaa993acc467a9672, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:29.079073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5ee42-f2de-40f1-a161-f2d9275b0475 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:29.087893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:29.093490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1008/2013] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:29 2022] PUT /identity/v3/projects/41584b9207a2459ebd0d0a8cab7c4a1b/users/d1d99a3a66014dabaa993acc467a9672/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:29.104178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2920c0-782e-457a-aa4f-e82095826b7f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.104463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2920c0-782e-457a-aa4f-e82095826b7f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.104738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2920c0-782e-457a-aa4f-e82095826b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:29.132641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-1e2920c0-782e-457a-aa4f-e82095826b7f None None] MFA Rules not processed for user `d1d99a3a66014dabaa993acc467a9672`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:29.133907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1e2920c0-782e-457a-aa4f-e82095826b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:29.137691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:29.211207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1006/2014] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:29.215439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5ee42-f2de-40f1-a161-f2d9275b0475 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:29.304609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5ee42-f2de-40f1-a161-f2d9275b0475 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:29.330157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b71c83-fb76-4832-84bd-368ffb9f1a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:29.331217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b71c83-fb76-4832-84bd-368ffb9f1a37 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.331704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b71c83-fb76-4832-84bd-368ffb9f1a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.332052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b71c83-fb76-4832-84bd-368ffb9f1a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:29.332853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b71c83-fb76-4832-84bd-368ffb9f1a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:29.335121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b71c83-fb76-4832-84bd-368ffb9f1a37 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:29.383940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1009/2015] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:29 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:29.401523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2920c0-782e-457a-aa4f-e82095826b7f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:29.404289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2920c0-782e-457a-aa4f-e82095826b7f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:29.482806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539276d-eb10-4105-a007-34af663c2265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:29.483464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539276d-eb10-4105-a007-34af663c2265 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.483730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539276d-eb10-4105-a007-34af663c2265 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.484157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539276d-eb10-4105-a007-34af663c2265 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:29.485376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539276d-eb10-4105-a007-34af663c2265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:29.493028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539276d-eb10-4105-a007-34af663c2265 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:29.521767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1007/2016] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:29 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:29.525543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b71c83-fb76-4832-84bd-368ffb9f1a37 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:29.536782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b71c83-fb76-4832-84bd-368ffb9f1a37 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:29.542027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e386a-2208-407b-90b4-acc8b7420774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:29.542838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e386a-2208-407b-90b4-acc8b7420774 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.543308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e386a-2208-407b-90b4-acc8b7420774 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.543783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e386a-2208-407b-90b4-acc8b7420774 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:29.544686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e386a-2208-407b-90b4-acc8b7420774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:29.546816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e386a-2208-407b-90b4-acc8b7420774 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:29.561614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1010/2017] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:29.562707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539276d-eb10-4105-a007-34af663c2265 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:29.573159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539276d-eb10-4105-a007-34af663c2265 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:29.578449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50022f-70fc-469b-9691-b41e67415498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:29.579102 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50022f-70fc-469b-9691-b41e67415498 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.579377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50022f-70fc-469b-9691-b41e67415498 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.579645 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50022f-70fc-469b-9691-b41e67415498 admin admin] PATH_INFO: `/v3/projects/de9230c05ea043f8924ef6e01fdfaed6/users/b30747099f4040caa2041a888a4e5988/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:29.594888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50022f-70fc-469b-9691-b41e67415498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de9230c05ea043f8924ef6e01fdfaed6, user_id=b30747099f4040caa2041a888a4e5988, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:29.598241 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50022f-70fc-469b-9691-b41e67415498 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:29.621194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1008/2018] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:29 2022] PUT /identity/v3/projects/de9230c05ea043f8924ef6e01fdfaed6/users/b30747099f4040caa2041a888a4e5988/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:53:29.622419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e386a-2208-407b-90b4-acc8b7420774 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:29.633292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e386a-2208-407b-90b4-acc8b7420774 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:29.700143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a106f-8b2e-4164-ab18-d7c6dd0611c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:29.700810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a106f-8b2e-4164-ab18-d7c6dd0611c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.701092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a106f-8b2e-4164-ab18-d7c6dd0611c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.701363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a106f-8b2e-4164-ab18-d7c6dd0611c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:29.702041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a106f-8b2e-4164-ab18-d7c6dd0611c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:29.703752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a106f-8b2e-4164-ab18-d7c6dd0611c7 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:29.718692 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1011/2019] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:29.721562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50022f-70fc-469b-9691-b41e67415498 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:29.735213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50022f-70fc-469b-9691-b41e67415498 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:29.740176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5997c51-0edb-46dc-beae-6724621ec643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:29.741534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5997c51-0edb-46dc-beae-6724621ec643 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.741534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5997c51-0edb-46dc-beae-6724621ec643 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.741534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5997c51-0edb-46dc-beae-6724621ec643 admin admin] PATH_INFO: `/v3/projects/de9230c05ea043f8924ef6e01fdfaed6/users/b30747099f4040caa2041a888a4e5988/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:29.743579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5997c51-0edb-46dc-beae-6724621ec643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de9230c05ea043f8924ef6e01fdfaed6, user_id=b30747099f4040caa2041a888a4e5988, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:29.748924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5997c51-0edb-46dc-beae-6724621ec643 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:29.766752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1009/2020] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:29 2022] PUT /identity/v3/projects/de9230c05ea043f8924ef6e01fdfaed6/users/b30747099f4040caa2041a888a4e5988/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:29.774261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d6ae6-1dd1-4496-9385-af01f7c003bf None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.775029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d6ae6-1dd1-4496-9385-af01f7c003bf None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.775437 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d6ae6-1dd1-4496-9385-af01f7c003bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:29.814603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f79d6ae6-1dd1-4496-9385-af01f7c003bf None None] MFA Rules not processed for user `b30747099f4040caa2041a888a4e5988`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:29.815826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f79d6ae6-1dd1-4496-9385-af01f7c003bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:29.901296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:29.915264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1012/2021] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:29.927516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5997c51-0edb-46dc-beae-6724621ec643 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:29.929568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d5997c51-0edb-46dc-beae-6724621ec643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:29.989212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5997c51-0edb-46dc-beae-6724621ec643 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:29.992851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd641b-7105-408e-a3d8-7cc0d2496340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:29.993455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd641b-7105-408e-a3d8-7cc0d2496340 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:29.993764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd641b-7105-408e-a3d8-7cc0d2496340 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:29.994039 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd641b-7105-408e-a3d8-7cc0d2496340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:30.003061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:30.050303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd641b-7105-408e-a3d8-7cc0d2496340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:30.051700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd641b-7105-408e-a3d8-7cc0d2496340 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:30.060653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1010/2022] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:31.580679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d6ae6-1dd1-4496-9385-af01f7c003bf None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:31.582234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f79d6ae6-1dd1-4496-9385-af01f7c003bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:31.595735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d6ae6-1dd1-4496-9385-af01f7c003bf None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:31.599343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675efe76-712c-4eca-81b5-882ce8830500 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:31.599956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675efe76-712c-4eca-81b5-882ce8830500 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:31.600236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675efe76-712c-4eca-81b5-882ce8830500 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:31.600508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675efe76-712c-4eca-81b5-882ce8830500 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:31.660515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675efe76-712c-4eca-81b5-882ce8830500 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:31.661870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675efe76-712c-4eca-81b5-882ce8830500 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:31.671629 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1013/2023] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:53:32.339814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd641b-7105-408e-a3d8-7cc0d2496340 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:32.350568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd641b-7105-408e-a3d8-7cc0d2496340 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:32.354381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a5bc6-7c64-4fa7-828e-03c2753e141b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:32.355288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a5bc6-7c64-4fa7-828e-03c2753e141b service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:32.355642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a5bc6-7c64-4fa7-828e-03c2753e141b service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:32.355986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a5bc6-7c64-4fa7-828e-03c2753e141b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:32.367058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a5bc6-7c64-4fa7-828e-03c2753e141b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:32.368777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a5bc6-7c64-4fa7-828e-03c2753e141b service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:32.392050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1011/2024] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:36.310623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675efe76-712c-4eca-81b5-882ce8830500 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:36.370586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675efe76-712c-4eca-81b5-882ce8830500 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:36.375027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:36.375640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:36.375918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:36.376181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] PATH_INFO: `/v3/users/78fdacaa8cc14d31a9d98cb543ab305c` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:36.377274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78fdacaa8cc14d31a9d98cb543ab305c)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:36.379207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:36.428201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78fdacaa8cc14d31a9d98cb543ab305c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:36.428862 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78fdacaa8cc14d31a9d98cb543ab305c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:36.435980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78fdacaa8cc14d31a9d98cb543ab305c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:36.447931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78fdacaa8cc14d31a9d98cb543ab305c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:36.457815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78fdacaa8cc14d31a9d98cb543ab305c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:36.463911 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1014/2025] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:36 2022] DELETE /identity/v3/users/78fdacaa8cc14d31a9d98cb543ab305c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:36.469303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddc575-42a5-4554-a15f-341a990d3275 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:36.469781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddc575-42a5-4554-a15f-341a990d3275 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:36.469895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddc575-42a5-4554-a15f-341a990d3275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:36.489000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ebddc575-42a5-4554-a15f-341a990d3275 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:36.490263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ebddc575-42a5-4554-a15f-341a990d3275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:36.553230 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1012/2026] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:36.564078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:36.565524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:36.580138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:36.623042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56951-2698-4cc0-b1fc-b0719a09cba5 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:36.627197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab7c74-a960-43bb-9ae1-a7ec0487607f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:36.627805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab7c74-a960-43bb-9ae1-a7ec0487607f service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:36.628085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab7c74-a960-43bb-9ae1-a7ec0487607f service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:36.628357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab7c74-a960-43bb-9ae1-a7ec0487607f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:36.637171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab7c74-a960-43bb-9ae1-a7ec0487607f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:36.638633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab7c74-a960-43bb-9ae1-a7ec0487607f service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:36.648488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1015/2027] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:36.909288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddc575-42a5-4554-a15f-341a990d3275 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:36.910756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ebddc575-42a5-4554-a15f-341a990d3275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:36.923328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddc575-42a5-4554-a15f-341a990d3275 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:36.928052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:36.928757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:36.929113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:36.929452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] PATH_INFO: `/v3/projects/ea045c28d6c848b9a49a73347c9b9e33` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:36.930704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea045c28d6c848b9a49a73347c9b9e33)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:36.932818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:36.979160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea045c28d6c848b9a49a73347c9b9e33', 'request_id': 'req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:36.986163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea045c28d6c848b9a49a73347c9b9e33', 'request_id': 'req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:36.992616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea045c28d6c848b9a49a73347c9b9e33', 'request_id': 'req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.003573 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] The token cache is being invalidate because project ea045c28d6c848b9a49a73347c9b9e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:37.003857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.004502 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.006036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1013/2028] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:36 2022] DELETE /identity/v3/projects/ea045c28d6c848b9a49a73347c9b9e33 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:37.010078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab7c74-a960-43bb-9ae1-a7ec0487607f service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:37.019873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab7c74-a960-43bb-9ae1-a7ec0487607f service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:37.075930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:37.076582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:37.076862 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:37.077136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] PATH_INFO: `/v3/users/d1d99a3a66014dabaa993acc467a9672` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:37.078340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d99a3a66014dabaa993acc467a9672)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:37.080392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:37.132897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d99a3a66014dabaa993acc467a9672'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.133580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d99a3a66014dabaa993acc467a9672'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.140323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d99a3a66014dabaa993acc467a9672'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.152395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d99a3a66014dabaa993acc467a9672'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.159426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d99a3a66014dabaa993acc467a9672'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.165288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1016/2029] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:37 2022] DELETE /identity/v3/users/d1d99a3a66014dabaa993acc467a9672 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:37.452240 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:37.454238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:37.467699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061cb5f-dc37-4272-a7e7-a1adccc6c2a0 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:37.488220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:37.533356 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:37.533936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:37.534217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:37.534489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] PATH_INFO: `/v3/projects/41584b9207a2459ebd0d0a8cab7c4a1b` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:37.535618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=41584b9207a2459ebd0d0a8cab7c4a1b)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:37.537525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:37.584112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41584b9207a2459ebd0d0a8cab7c4a1b', 'request_id': 'req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.592590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41584b9207a2459ebd0d0a8cab7c4a1b', 'request_id': 'req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.599245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41584b9207a2459ebd0d0a8cab7c4a1b', 'request_id': 'req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.610372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] The token cache is being invalidate because project 41584b9207a2459ebd0d0a8cab7c4a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:37.610707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.611354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.613163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1014/2030] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:37 2022] DELETE /identity/v3/projects/41584b9207a2459ebd0d0a8cab7c4a1b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:37.616969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:37.618432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:37.631678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ab37-2441-45b0-a714-ac0cbab81f75 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:37.665348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:37.699552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:37.700201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:37.700478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:37.700746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] PATH_INFO: `/v3/users/b30747099f4040caa2041a888a4e5988` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:37.702433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30747099f4040caa2041a888a4e5988)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:37.704351 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:37.762371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30747099f4040caa2041a888a4e5988'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.763033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30747099f4040caa2041a888a4e5988'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.769866 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30747099f4040caa2041a888a4e5988'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.781566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30747099f4040caa2041a888a4e5988'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.788887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-bb02b2e0-cc11-45f5-8a94-305a77d7d331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30747099f4040caa2041a888a4e5988'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:37.795058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1017/2031] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:37 2022] DELETE /identity/v3/users/b30747099f4040caa2041a888a4e5988 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:53:38.075501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:38.076955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:38.090377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58accb91-dc12-4f1c-b2da-9fdfe07c0dec admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:38.149290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:38.149944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.150219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.150485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] PATH_INFO: `/v3/projects/de9230c05ea043f8924ef6e01fdfaed6` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.151588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] RBAC: Authorizing `identity:delete_project(project_id=de9230c05ea043f8924ef6e01fdfaed6)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:38.153477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:38.206662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de9230c05ea043f8924ef6e01fdfaed6', 'request_id': 'req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:38.210603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:38.217983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de9230c05ea043f8924ef6e01fdfaed6', 'request_id': 'req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:38.225431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de9230c05ea043f8924ef6e01fdfaed6', 'request_id': 'req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:38.236743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] The token cache is being invalidate because project de9230c05ea043f8924ef6e01fdfaed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:38.237026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:38.237706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2bc8e403-0011-4e2e-8f9d-235a8cf4705a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:38.239183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1015/2032] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:38 2022] DELETE /identity/v3/projects/de9230c05ea043f8924ef6e01fdfaed6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:38.244028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5e16d-1a2e-4098-9644-1976eda0dcf7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.244320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5e16d-1a2e-4098-9644-1976eda0dcf7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.244591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5e16d-1a2e-4098-9644-1976eda0dcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.263768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e3a5e16d-1a2e-4098-9644-1976eda0dcf7 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:38.264972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e3a5e16d-1a2e-4098-9644-1976eda0dcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:38.324846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1018/2033] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:38.332506 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760279e-9b5a-427d-a996-875579ce12e8 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.332786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760279e-9b5a-427d-a996-875579ce12e8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.333103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760279e-9b5a-427d-a996-875579ce12e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.353722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b760279e-9b5a-427d-a996-875579ce12e8 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:38.354948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b760279e-9b5a-427d-a996-875579ce12e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:38.363412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1016/2034] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:38.367492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5e16d-1a2e-4098-9644-1976eda0dcf7 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:38.378056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5e16d-1a2e-4098-9644-1976eda0dcf7 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:38.383421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d1351-29c3-4952-85f6-59735b8e625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:38.384031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d1351-29c3-4952-85f6-59735b8e625f admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.384309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d1351-29c3-4952-85f6-59735b8e625f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.384600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d1351-29c3-4952-85f6-59735b8e625f admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.385248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d1351-29c3-4952-85f6-59735b8e625f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:53:38.385578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d1351-29c3-4952-85f6-59735b8e625f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:38.387175 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d1351-29c3-4952-85f6-59735b8e625f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:38.399422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1019/2035] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:53:38.404056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760279e-9b5a-427d-a996-875579ce12e8 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:38.414229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760279e-9b5a-427d-a996-875579ce12e8 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:38.418880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395da9e-5253-4d8a-94a3-7b4d81a63504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:38.419487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395da9e-5253-4d8a-94a3-7b4d81a63504 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.419760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395da9e-5253-4d8a-94a3-7b4d81a63504 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.420026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395da9e-5253-4d8a-94a3-7b4d81a63504 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.421209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395da9e-5253-4d8a-94a3-7b4d81a63504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:38.423097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395da9e-5253-4d8a-94a3-7b4d81a63504 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:38.456121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1017/2036] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:38 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:38.460244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d1351-29c3-4952-85f6-59735b8e625f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:38.472638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d1351-29c3-4952-85f6-59735b8e625f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:38.540481 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:38.581196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ff0f8-39dc-4fe2-b2e2-8754fa0ad819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:38.581860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ff0f8-39dc-4fe2-b2e2-8754fa0ad819 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.582143 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ff0f8-39dc-4fe2-b2e2-8754fa0ad819 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.582413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ff0f8-39dc-4fe2-b2e2-8754fa0ad819 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.583114 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ff0f8-39dc-4fe2-b2e2-8754fa0ad819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:38.584976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ff0f8-39dc-4fe2-b2e2-8754fa0ad819 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:38.626184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1020/2037] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:38 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:38.630019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395da9e-5253-4d8a-94a3-7b4d81a63504 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:38.654129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395da9e-5253-4d8a-94a3-7b4d81a63504 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:38.659792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db3bd3-3a04-47b2-897a-72d40d5029c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:38.660527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db3bd3-3a04-47b2-897a-72d40d5029c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.660902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db3bd3-3a04-47b2-897a-72d40d5029c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.661243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db3bd3-3a04-47b2-897a-72d40d5029c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.662100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db3bd3-3a04-47b2-897a-72d40d5029c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:38.663790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db3bd3-3a04-47b2-897a-72d40d5029c5 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:38.674073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1018/2038] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:38.678144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ff0f8-39dc-4fe2-b2e2-8754fa0ad819 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:38.688571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ff0f8-39dc-4fe2-b2e2-8754fa0ad819 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:38.693312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747d52a-59a7-47fa-a805-c04390b42398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:38.693967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747d52a-59a7-47fa-a805-c04390b42398 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.694249 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747d52a-59a7-47fa-a805-c04390b42398 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.694524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747d52a-59a7-47fa-a805-c04390b42398 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.695181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747d52a-59a7-47fa-a805-c04390b42398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:38.696797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747d52a-59a7-47fa-a805-c04390b42398 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:38.709322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1021/2039] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:38.713781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db3bd3-3a04-47b2-897a-72d40d5029c5 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:38.725602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db3bd3-3a04-47b2-897a-72d40d5029c5 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:38.730505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25de538-4dd0-4249-be2a-fb33d5d47c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:38.731155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25de538-4dd0-4249-be2a-fb33d5d47c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.731460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25de538-4dd0-4249-be2a-fb33d5d47c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.731731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25de538-4dd0-4249-be2a-fb33d5d47c52 admin admin] PATH_INFO: `/v3/projects/a3cf9f0fb1234fbcb0156092edd34bb5/users/e678b8201f534d999ebfd7c2ecb836f7/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.748440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25de538-4dd0-4249-be2a-fb33d5d47c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cf9f0fb1234fbcb0156092edd34bb5, user_id=e678b8201f534d999ebfd7c2ecb836f7, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:38.751674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25de538-4dd0-4249-be2a-fb33d5d47c52 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:38.770208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1019/2040] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:38 2022] PUT /identity/v3/projects/a3cf9f0fb1234fbcb0156092edd34bb5/users/e678b8201f534d999ebfd7c2ecb836f7/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:38.780864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7832741-ee7a-4bfe-bb33-49bb5794ed99 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.781126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7832741-ee7a-4bfe-bb33-49bb5794ed99 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.781413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7832741-ee7a-4bfe-bb33-49bb5794ed99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.812066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e7832741-ee7a-4bfe-bb33-49bb5794ed99 None None] MFA Rules not processed for user `e678b8201f534d999ebfd7c2ecb836f7`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:38.815083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e7832741-ee7a-4bfe-bb33-49bb5794ed99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:38.886380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1022/2041] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:38.895172 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25de538-4dd0-4249-be2a-fb33d5d47c52 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:38.896839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c25de538-4dd0-4249-be2a-fb33d5d47c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:38.976723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25de538-4dd0-4249-be2a-fb33d5d47c52 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:38.980389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f515e-3b26-4af1-b972-4395a89cf6e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd04df68f9d84fc4bb981c8b69cc771a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:38.981003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f515e-3b26-4af1-b972-4395a89cf6e4 service glance] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:38.981282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f515e-3b26-4af1-b972-4395a89cf6e4 service glance] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:38.981591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f515e-3b26-4af1-b972-4395a89cf6e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:38.991134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f515e-3b26-4af1-b972-4395a89cf6e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:38.992670 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f515e-3b26-4af1-b972-4395a89cf6e4 service glance] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:39.003040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1020/2042] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:40.993199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15e3f7-fd2c-4cb0-ad5f-85cd6e200557 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:40.993799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15e3f7-fd2c-4cb0-ad5f-85cd6e200557 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:40.994444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15e3f7-fd2c-4cb0-ad5f-85cd6e200557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:41.024309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-5d15e3f7-fd2c-4cb0-ad5f-85cd6e200557 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:41.026541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5d15e3f7-fd2c-4cb0-ad5f-85cd6e200557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:41.087166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1023/2043] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 14:53:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:41.131022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f515e-3b26-4af1-b972-4395a89cf6e4 service glance] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:41.132419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f80f515e-3b26-4af1-b972-4395a89cf6e4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:41.197115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:41.202157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f515e-3b26-4af1-b972-4395a89cf6e4 service glance] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:41.208625 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f3252-a770-40f4-8863-9e8d4b883bc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:41.210106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f3252-a770-40f4-8863-9e8d4b883bc1 service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:41.210438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f3252-a770-40f4-8863-9e8d4b883bc1 service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:41.210724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f3252-a770-40f4-8863-9e8d4b883bc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:41.225789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f3252-a770-40f4-8863-9e8d4b883bc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:41.227502 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f3252-a770-40f4-8863-9e8d4b883bc1 service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:41.237006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1021/2044] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 14:53:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 14:53:41.840178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95001fa-db27-4aa4-bf08-4f9856e5601b None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:41.840485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95001fa-db27-4aa4-bf08-4f9856e5601b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:41.840763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95001fa-db27-4aa4-bf08-4f9856e5601b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:41.860472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f95001fa-db27-4aa4-bf08-4f9856e5601b None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:41.861728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f95001fa-db27-4aa4-bf08-4f9856e5601b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:41.868914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1024/2045] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 3 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:41.882067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f3252-a770-40f4-8863-9e8d4b883bc1 service swift] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:41.892846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f3252-a770-40f4-8863-9e8d4b883bc1 service swift] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:41.897854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bfd71-9595-4ab7-a1bf-88db1c1da90d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:41.898449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bfd71-9595-4ab7-a1bf-88db1c1da90d service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:41.898730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bfd71-9595-4ab7-a1bf-88db1c1da90d service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:41.899000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bfd71-9595-4ab7-a1bf-88db1c1da90d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:41.907323 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bfd71-9595-4ab7-a1bf-88db1c1da90d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:41.908927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bfd71-9595-4ab7-a1bf-88db1c1da90d service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:41.916898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1022/2046] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 14:53:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 03 14:53:42.158415 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6972687-6ec5-4f42-8479-e41ab6b7bf61 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:42.158931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6972687-6ec5-4f42-8479-e41ab6b7bf61 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:42.159171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6972687-6ec5-4f42-8479-e41ab6b7bf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:42.179520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f6972687-6ec5-4f42-8479-e41ab6b7bf61 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:42.180727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f6972687-6ec5-4f42-8479-e41ab6b7bf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:42.188804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1025/2047] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:42.202464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bfd71-9595-4ab7-a1bf-88db1c1da90d service swift] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:42.213799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bfd71-9595-4ab7-a1bf-88db1c1da90d service swift] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:42.219855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74238a-f55e-4d8a-aa42-d39da1976d4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:42.220837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74238a-f55e-4d8a-aa42-d39da1976d4b service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:42.221290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74238a-f55e-4d8a-aa42-d39da1976d4b service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:42.221664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74238a-f55e-4d8a-aa42-d39da1976d4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:42.231897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74238a-f55e-4d8a-aa42-d39da1976d4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:42.233509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74238a-f55e-4d8a-aa42-d39da1976d4b service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:42.242967 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1023/2048] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 14:53:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 03 14:53:42.619552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c812c-6cdd-4166-bc4f-84377512b2cd None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:42.619847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c812c-6cdd-4166-bc4f-84377512b2cd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:42.620125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c812c-6cdd-4166-bc4f-84377512b2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:42.641147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e81c812c-6cdd-4166-bc4f-84377512b2cd None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:42.642403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e81c812c-6cdd-4166-bc4f-84377512b2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:42.650426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1026/2049] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 3 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:42.663467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74238a-f55e-4d8a-aa42-d39da1976d4b service swift] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:42.674566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74238a-f55e-4d8a-aa42-d39da1976d4b service swift] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:42.680385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b1f7a-88a9-443e-bc75-5075a792c6eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:42.680991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b1f7a-88a9-443e-bc75-5075a792c6eb service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:42.681273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b1f7a-88a9-443e-bc75-5075a792c6eb service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:42.681566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b1f7a-88a9-443e-bc75-5075a792c6eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:42.689875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b1f7a-88a9-443e-bc75-5075a792c6eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:42.691462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b1f7a-88a9-443e-bc75-5075a792c6eb service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:42.700263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1024/2050] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 14:53:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 14:53:44.744337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849e33c-d36c-4037-b0b8-0fbe749ccaa1 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:44.744880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849e33c-d36c-4037-b0b8-0fbe749ccaa1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:44.744880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849e33c-d36c-4037-b0b8-0fbe749ccaa1 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:44.746302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1027/2051] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:53:44.758508 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf8f5-3971-471a-9b8e-6aaf531c702c None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:44.758950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf8f5-3971-471a-9b8e-6aaf531c702c None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:44.759217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf8f5-3971-471a-9b8e-6aaf531c702c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:44.778921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c02cf8f5-3971-471a-9b8e-6aaf531c702c None None] MFA Rules not processed for user `ebc01c87551044818e1ccbb46b8e7340`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:44.780115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c02cf8f5-3971-471a-9b8e-6aaf531c702c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:44.817875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849e33c-d36c-4037-b0b8-0fbe749ccaa1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:44.820144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b849e33c-d36c-4037-b0b8-0fbe749ccaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:44.855373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1025/2052] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:53:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:44.870962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cf8f5-3971-471a-9b8e-6aaf531c702c None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:44.874859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c02cf8f5-3971-471a-9b8e-6aaf531c702c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:44.889029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cf8f5-3971-471a-9b8e-6aaf531c702c None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:44.889445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:44.897142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c5c6b-cd20-4e2c-84e3-5c94a56cc64e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:44.898042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c5c6b-cd20-4e2c-84e3-5c94a56cc64e service placement] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:44.898397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c5c6b-cd20-4e2c-84e3-5c94a56cc64e service placement] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:44.898955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c5c6b-cd20-4e2c-84e3-5c94a56cc64e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:44.909165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c5c6b-cd20-4e2c-84e3-5c94a56cc64e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:44.922279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c5c6b-cd20-4e2c-84e3-5c94a56cc64e service placement] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:44.938226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849e33c-d36c-4037-b0b8-0fbe749ccaa1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:44.944699 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:44.945651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:44.945932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:44.946204 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] PATH_INFO: `/v3/users/913c969eb6274750aef880400a74e637` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:44.947424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] RBAC: Authorizing `identity:delete_user(user_id=913c969eb6274750aef880400a74e637)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:44.949379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:44.949784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1026/2053] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:44.997355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913c969eb6274750aef880400a74e637'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:44.998046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913c969eb6274750aef880400a74e637'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.005638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913c969eb6274750aef880400a74e637'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.017755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913c969eb6274750aef880400a74e637'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.024648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913c969eb6274750aef880400a74e637'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.030834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1028/2054] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:44 2022] DELETE /identity/v3/users/913c969eb6274750aef880400a74e637 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:45.203860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:45.204018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c5c6b-cd20-4e2c-84e3-5c94a56cc64e service placement] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:45.205727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-738c5c6b-cd20-4e2c-84e3-5c94a56cc64e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:45.205853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:45.247815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c5c6b-cd20-4e2c-84e3-5c94a56cc64e service placement] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:45.293684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba08b2a-403c-4579-856a-cd7aad5ccfba admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:45.299590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:45.300347 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.300633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.300910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] PATH_INFO: `/v3/projects/2e3b8c8420474d9397a3578350b9eacf` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.302164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e3b8c8420474d9397a3578350b9eacf)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:45.304090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:45.310868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:45.311517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.312486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.312765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] PATH_INFO: `/v3/users/fc880a31f5be4ea2a500bfa504749ce0` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.313942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc880a31f5be4ea2a500bfa504749ce0)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:45.316653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:45.358476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e3b8c8420474d9397a3578350b9eacf', 'request_id': 'req-648fd5e2-33c2-430b-b60e-0749b23c6581'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.361216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:45.372614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc880a31f5be4ea2a500bfa504749ce0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.373283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc880a31f5be4ea2a500bfa504749ce0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.381188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc880a31f5be4ea2a500bfa504749ce0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.381372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e3b8c8420474d9397a3578350b9eacf', 'request_id': 'req-648fd5e2-33c2-430b-b60e-0749b23c6581'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.383080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:45.389893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e3b8c8420474d9397a3578350b9eacf', 'request_id': 'req-648fd5e2-33c2-430b-b60e-0749b23c6581'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.391348 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc880a31f5be4ea2a500bfa504749ce0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.392259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-04e3138a-c419-40a8-980b-fab7c68fddeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc880a31f5be4ea2a500bfa504749ce0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.404162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] The token cache is being invalidate because project 2e3b8c8420474d9397a3578350b9eacf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:45.404455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.405164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-648fd5e2-33c2-430b-b60e-0749b23c6581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.406890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1029/2055] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:45 2022] DELETE /identity/v3/projects/2e3b8c8420474d9397a3578350b9eacf => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:45.414660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1027/2056] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:45 2022] DELETE /identity/v3/users/fc880a31f5be4ea2a500bfa504749ce0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:45.417441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed1171-712e-483c-88a7-18667a77ffe3 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.417798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed1171-712e-483c-88a7-18667a77ffe3 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.418207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed1171-712e-483c-88a7-18667a77ffe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.419489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b6bd2-a0b9-47c0-86f6-00021efc3cbf None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.419780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b6bd2-a0b9-47c0-86f6-00021efc3cbf None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.420059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b6bd2-a0b9-47c0-86f6-00021efc3cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.440337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e0ed1171-712e-483c-88a7-18667a77ffe3 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:45.441764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e0ed1171-712e-483c-88a7-18667a77ffe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:45.442513 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ee2b6bd2-a0b9-47c0-86f6-00021efc3cbf None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:45.443816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ee2b6bd2-a0b9-47c0-86f6-00021efc3cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:45.518804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1030/2057] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:45.524022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1028/2058] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:45.531670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027e0ab-b9b7-4e4f-a1b6-53caf9dff52f None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.531964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027e0ab-b9b7-4e4f-a1b6-53caf9dff52f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.532245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027e0ab-b9b7-4e4f-a1b6-53caf9dff52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.534583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b6bd2-a0b9-47c0-86f6-00021efc3cbf None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:45.536247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ee2b6bd2-a0b9-47c0-86f6-00021efc3cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:45.553084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d027e0ab-b9b7-4e4f-a1b6-53caf9dff52f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:45.554327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d027e0ab-b9b7-4e4f-a1b6-53caf9dff52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:45.563431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1031/2059] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:45.570628 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027e0ab-b9b7-4e4f-a1b6-53caf9dff52f None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:45.581100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027e0ab-b9b7-4e4f-a1b6-53caf9dff52f None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:45.586889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317e83a-c0c9-40f1-8ab7-45ebfccc6de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:45.587565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317e83a-c0c9-40f1-8ab7-45ebfccc6de6 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.587936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317e83a-c0c9-40f1-8ab7-45ebfccc6de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.588280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317e83a-c0c9-40f1-8ab7-45ebfccc6de6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.589000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317e83a-c0c9-40f1-8ab7-45ebfccc6de6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:53:45.589376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317e83a-c0c9-40f1-8ab7-45ebfccc6de6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:45.591014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317e83a-c0c9-40f1-8ab7-45ebfccc6de6 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:45.602953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b6bd2-a0b9-47c0-86f6-00021efc3cbf None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:45.603640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1032/2060] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:53:45.606557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9894ef-bfea-4a69-a408-26698119463a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:45.607160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9894ef-bfea-4a69-a408-26698119463a service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.607441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9894ef-bfea-4a69-a408-26698119463a service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.607716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9894ef-bfea-4a69-a408-26698119463a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.607843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317e83a-c0c9-40f1-8ab7-45ebfccc6de6 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:45.617652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9894ef-bfea-4a69-a408-26698119463a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:45.618854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317e83a-c0c9-40f1-8ab7-45ebfccc6de6 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:45.619374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9894ef-bfea-4a69-a408-26698119463a service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:45.623393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e322c15-a87c-40fa-b17a-29a55ca7de66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:45.623987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e322c15-a87c-40fa-b17a-29a55ca7de66 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.624296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e322c15-a87c-40fa-b17a-29a55ca7de66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.624568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e322c15-a87c-40fa-b17a-29a55ca7de66 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.625187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e322c15-a87c-40fa-b17a-29a55ca7de66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:45.627090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e322c15-a87c-40fa-b17a-29a55ca7de66 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:45.629424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1029/2061] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:53:45.658476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1033/2062] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:45 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:45.659660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9894ef-bfea-4a69-a408-26698119463a service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:45.669191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9894ef-bfea-4a69-a408-26698119463a service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:45.721601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:45.728775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083ccd7-d445-4649-866f-42db4ff2e7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:45.729424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083ccd7-d445-4649-866f-42db4ff2e7ab admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.730082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083ccd7-d445-4649-866f-42db4ff2e7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.730402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083ccd7-d445-4649-866f-42db4ff2e7ab admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.731174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083ccd7-d445-4649-866f-42db4ff2e7ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:45.733527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083ccd7-d445-4649-866f-42db4ff2e7ab admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:45.758333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1030/2063] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:45 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:45.762410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e322c15-a87c-40fa-b17a-29a55ca7de66 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:45.774422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e322c15-a87c-40fa-b17a-29a55ca7de66 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:45.779510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a0f0c-da2d-419e-aea8-f277e2d6431c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:45.780122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0f0c-da2d-419e-aea8-f277e2d6431c admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.780433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0f0c-da2d-419e-aea8-f277e2d6431c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.781359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0f0c-da2d-419e-aea8-f277e2d6431c admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.782055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a0f0c-da2d-419e-aea8-f277e2d6431c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:45.783669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a0f0c-da2d-419e-aea8-f277e2d6431c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:45.794422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1034/2064] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:45.799483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083ccd7-d445-4649-866f-42db4ff2e7ab admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:45.814595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a0f0c-da2d-419e-aea8-f277e2d6431c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:45.815980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083ccd7-d445-4649-866f-42db4ff2e7ab admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:45.822397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1732d50-f0f3-404b-b7d9-efb39141b2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:45.824467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-426a0f0c-da2d-419e-aea8-f277e2d6431c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:45.828913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1732d50-f0f3-404b-b7d9-efb39141b2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.829202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1732d50-f0f3-404b-b7d9-efb39141b2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.829492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1732d50-f0f3-404b-b7d9-efb39141b2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.830209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1732d50-f0f3-404b-b7d9-efb39141b2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:45.832032 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1732d50-f0f3-404b-b7d9-efb39141b2f8 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:45.843759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1031/2065] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:45.847762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1732d50-f0f3-404b-b7d9-efb39141b2f8 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:45.850798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a0f0c-da2d-419e-aea8-f277e2d6431c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:45.858469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:45.859876 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.860166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.860469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] PATH_INFO: `/v3/projects/2a3f1ef0d0c343dfbf2eae49c0ea450c` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.860614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1732d50-f0f3-404b-b7d9-efb39141b2f8 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:45.862759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a3f1ef0d0c343dfbf2eae49c0ea450c)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:45.865654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:45.865998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86e108-3b3b-4b31-ae45-4342da928691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:45.866662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86e108-3b3b-4b31-ae45-4342da928691 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.866946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86e108-3b3b-4b31-ae45-4342da928691 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.867222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86e108-3b3b-4b31-ae45-4342da928691 admin admin] PATH_INFO: `/v3/projects/eb178f7036624ee5acb4edeaa8c5192a/users/b70bc2ca226147f8b4fb2a709cfe70e0/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.883800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86e108-3b3b-4b31-ae45-4342da928691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb178f7036624ee5acb4edeaa8c5192a, user_id=b70bc2ca226147f8b4fb2a709cfe70e0, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:45.886654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86e108-3b3b-4b31-ae45-4342da928691 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:45.901475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1032/2066] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:45 2022] PUT /identity/v3/projects/eb178f7036624ee5acb4edeaa8c5192a/users/b70bc2ca226147f8b4fb2a709cfe70e0/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:45.912496 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c9326-7be0-42a0-9b5e-7f5daf5f2238 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.912772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c9326-7be0-42a0-9b5e-7f5daf5f2238 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.913040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c9326-7be0-42a0-9b5e-7f5daf5f2238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.917403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a3f1ef0d0c343dfbf2eae49c0ea450c', 'request_id': 'req-28096e18-9526-475f-bfd7-8e2eb4967977'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.928369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a3f1ef0d0c343dfbf2eae49c0ea450c', 'request_id': 'req-28096e18-9526-475f-bfd7-8e2eb4967977'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.937372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a3f1ef0d0c343dfbf2eae49c0ea450c', 'request_id': 'req-28096e18-9526-475f-bfd7-8e2eb4967977'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.943314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-4f7c9326-7be0-42a0-9b5e-7f5daf5f2238 None None] MFA Rules not processed for user `b70bc2ca226147f8b4fb2a709cfe70e0`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:45.944605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4f7c9326-7be0-42a0-9b5e-7f5daf5f2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:45.950682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] The token cache is being invalidate because project 2a3f1ef0d0c343dfbf2eae49c0ea450c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:45.951458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.952134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-28096e18-9526-475f-bfd7-8e2eb4967977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:45.955202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1035/2067] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:45 2022] DELETE /identity/v3/projects/2a3f1ef0d0c343dfbf2eae49c0ea450c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:53:45.960299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd6a3e-95da-4a36-8942-ee1768a44720 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:45.960647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd6a3e-95da-4a36-8942-ee1768a44720 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:45.960927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd6a3e-95da-4a36-8942-ee1768a44720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:45.992036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-45fd6a3e-95da-4a36-8942-ee1768a44720 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:45.993239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-45fd6a3e-95da-4a36-8942-ee1768a44720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:46.003069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:46.019588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1033/2068] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:46.024037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c9326-7be0-42a0-9b5e-7f5daf5f2238 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.025413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4f7c9326-7be0-42a0-9b5e-7f5daf5f2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:46.056643 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1036/2069] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:46.065327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec69d28-b6cc-4bd1-aae4-bc533109d709 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.065656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec69d28-b6cc-4bd1-aae4-bc533109d709 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.065937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec69d28-b6cc-4bd1-aae4-bc533109d709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.086332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2ec69d28-b6cc-4bd1-aae4-bc533109d709 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:46.087522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2ec69d28-b6cc-4bd1-aae4-bc533109d709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:46.091372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c9326-7be0-42a0-9b5e-7f5daf5f2238 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.096970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dcbe4-b154-47d2-bfa2-59ce7133886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.097285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1037/2070] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:46.101235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dcbe4-b154-47d2-bfa2-59ce7133886c admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.101587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dcbe4-b154-47d2-bfa2-59ce7133886c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.101847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dcbe4-b154-47d2-bfa2-59ce7133886c admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.102065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec69d28-b6cc-4bd1-aae4-bc533109d709 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.102628 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572dcbe4-b154-47d2-bfa2-59ce7133886c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.105015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572dcbe4-b154-47d2-bfa2-59ce7133886c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.116138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec69d28-b6cc-4bd1-aae4-bc533109d709 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.121585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f159e3-b66d-4b3b-ae60-ff03bfc312ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.122209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f159e3-b66d-4b3b-ae60-ff03bfc312ff admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.122497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f159e3-b66d-4b3b-ae60-ff03bfc312ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.122768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f159e3-b66d-4b3b-ae60-ff03bfc312ff admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.123523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f159e3-b66d-4b3b-ae60-ff03bfc312ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:53:46.123732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f159e3-b66d-4b3b-ae60-ff03bfc312ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.125582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f159e3-b66d-4b3b-ae60-ff03bfc312ff admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.134955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1034/2071] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:46 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:46.138141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1038/2072] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:53:46.139040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dcbe4-b154-47d2-bfa2-59ce7133886c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.142500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f159e3-b66d-4b3b-ae60-ff03bfc312ff admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.151786 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f159e3-b66d-4b3b-ae60-ff03bfc312ff admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.168716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:46.203391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dcbe4-b154-47d2-bfa2-59ce7133886c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.208013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663d618-26e4-4584-9a87-715a802b3008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.208612 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663d618-26e4-4584-9a87-715a802b3008 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.208888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663d618-26e4-4584-9a87-715a802b3008 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.208994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bc65c-c428-4fd3-b300-0680ad8c353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.209156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663d618-26e4-4584-9a87-715a802b3008 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.209548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bc65c-c428-4fd3-b300-0680ad8c353a admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.209819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663d618-26e4-4584-9a87-715a802b3008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.209909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bc65c-c428-4fd3-b300-0680ad8c353a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.210095 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bc65c-c428-4fd3-b300-0680ad8c353a admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.210745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bc65c-c428-4fd3-b300-0680ad8c353a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.211737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663d618-26e4-4584-9a87-715a802b3008 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.212655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bc65c-c428-4fd3-b300-0680ad8c353a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.236347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1035/2073] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:46 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:46.237379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663d618-26e4-4584-9a87-715a802b3008 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.247118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1039/2074] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:46 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:46.247269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663d618-26e4-4584-9a87-715a802b3008 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.248295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bc65c-c428-4fd3-b300-0680ad8c353a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.309662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79599c66-0916-49a5-8d08-c7f810ddec5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.310273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79599c66-0916-49a5-8d08-c7f810ddec5c admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.310554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79599c66-0916-49a5-8d08-c7f810ddec5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.310830 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79599c66-0916-49a5-8d08-c7f810ddec5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.311476 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79599c66-0916-49a5-8d08-c7f810ddec5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.313630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79599c66-0916-49a5-8d08-c7f810ddec5c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.313830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bc65c-c428-4fd3-b300-0680ad8c353a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.318524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3584d-ae9d-48cd-a87d-724ab9e533ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.319138 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3584d-ae9d-48cd-a87d-724ab9e533ce admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.319399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3584d-ae9d-48cd-a87d-724ab9e533ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.319663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3584d-ae9d-48cd-a87d-724ab9e533ce admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.320288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3584d-ae9d-48cd-a87d-724ab9e533ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.322217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3584d-ae9d-48cd-a87d-724ab9e533ce admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.323811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1036/2075] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:46.328614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79599c66-0916-49a5-8d08-c7f810ddec5c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.339592 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79599c66-0916-49a5-8d08-c7f810ddec5c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.344440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302234e8-fd86-40ad-89e7-cbc68390e79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.347661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1040/2076] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:46 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:46.347783 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302234e8-fd86-40ad-89e7-cbc68390e79e admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.347783 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302234e8-fd86-40ad-89e7-cbc68390e79e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.347783 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302234e8-fd86-40ad-89e7-cbc68390e79e admin admin] PATH_INFO: `/v3/projects/6f3412747e564785a902780e85555db4/users/8010953562b04d11a3da3ccaaf72714b/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.348848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3584d-ae9d-48cd-a87d-724ab9e533ce admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.359354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3584d-ae9d-48cd-a87d-724ab9e533ce admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.361941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302234e8-fd86-40ad-89e7-cbc68390e79e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3412747e564785a902780e85555db4, user_id=8010953562b04d11a3da3ccaaf72714b, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.364062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87782d97-2680-4cc3-925e-1951fc31606e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.364654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87782d97-2680-4cc3-925e-1951fc31606e admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.364795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302234e8-fd86-40ad-89e7-cbc68390e79e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.364922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87782d97-2680-4cc3-925e-1951fc31606e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.365201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87782d97-2680-4cc3-925e-1951fc31606e admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.365902 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87782d97-2680-4cc3-925e-1951fc31606e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.367486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87782d97-2680-4cc3-925e-1951fc31606e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.373575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:46.379322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1041/2077] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:46.379882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1037/2078] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:46 2022] PUT /identity/v3/projects/6f3412747e564785a902780e85555db4/users/8010953562b04d11a3da3ccaaf72714b/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:53:46.383239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302234e8-fd86-40ad-89e7-cbc68390e79e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.383723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87782d97-2680-4cc3-925e-1951fc31606e admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.394084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87782d97-2680-4cc3-925e-1951fc31606e admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.453732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302234e8-fd86-40ad-89e7-cbc68390e79e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.458436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf07f3-4f87-4193-b5c0-4600f6403623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.459037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf07f3-4f87-4193-b5c0-4600f6403623 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.459319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf07f3-4f87-4193-b5c0-4600f6403623 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.459533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4445a2-ad3d-48b5-b511-626ee64fb99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.459652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf07f3-4f87-4193-b5c0-4600f6403623 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.460135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4445a2-ad3d-48b5-b511-626ee64fb99b admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.460225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf07f3-4f87-4193-b5c0-4600f6403623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.460413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4445a2-ad3d-48b5-b511-626ee64fb99b admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.460686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4445a2-ad3d-48b5-b511-626ee64fb99b admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.461322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4445a2-ad3d-48b5-b511-626ee64fb99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.462388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf07f3-4f87-4193-b5c0-4600f6403623 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.463205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4445a2-ad3d-48b5-b511-626ee64fb99b admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.472305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1038/2079] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:53:46.473122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1042/2080] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:53:46.476341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf07f3-4f87-4193-b5c0-4600f6403623 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.477102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4445a2-ad3d-48b5-b511-626ee64fb99b admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.486308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf07f3-4f87-4193-b5c0-4600f6403623 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.486484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4445a2-ad3d-48b5-b511-626ee64fb99b admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.490963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27d450-400c-4987-996e-bc23df05d769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.491110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af5858-315e-4a61-9136-d6c21d51d7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.491595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27d450-400c-4987-996e-bc23df05d769 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.491702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af5858-315e-4a61-9136-d6c21d51d7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.491870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27d450-400c-4987-996e-bc23df05d769 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.491973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af5858-315e-4a61-9136-d6c21d51d7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.492142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27d450-400c-4987-996e-bc23df05d769 admin admin] PATH_INFO: `/v3/projects/6f3412747e564785a902780e85555db4/users/8010953562b04d11a3da3ccaaf72714b/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.492311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af5858-315e-4a61-9136-d6c21d51d7e5 admin admin] PATH_INFO: `/v3/projects/a167b2ccacbc4e6d903b09d9a00650fe/users/7cfae014351641d8a949775875a45958/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.494344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27d450-400c-4987-996e-bc23df05d769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3412747e564785a902780e85555db4, user_id=8010953562b04d11a3da3ccaaf72714b, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.496976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27d450-400c-4987-996e-bc23df05d769 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.507789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af5858-315e-4a61-9136-d6c21d51d7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a167b2ccacbc4e6d903b09d9a00650fe, user_id=7cfae014351641d8a949775875a45958, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.509220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1043/2081] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:46 2022] PUT /identity/v3/projects/6f3412747e564785a902780e85555db4/users/8010953562b04d11a3da3ccaaf72714b/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:53:46.520467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af5858-315e-4a61-9136-d6c21d51d7e5 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.520467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:46.523137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a264df5-ed6b-45c0-8fab-a4c05ada4786 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.523496 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a264df5-ed6b-45c0-8fab-a4c05ada4786 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.523787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a264df5-ed6b-45c0-8fab-a4c05ada4786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.525944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1039/2082] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:46 2022] PUT /identity/v3/projects/a167b2ccacbc4e6d903b09d9a00650fe/users/7cfae014351641d8a949775875a45958/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:53:46.530564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6220da-cbe2-4231-aad7-8f1a5695c27b None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.530894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6220da-cbe2-4231-aad7-8f1a5695c27b None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.531171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6220da-cbe2-4231-aad7-8f1a5695c27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.549153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-2f6220da-cbe2-4231-aad7-8f1a5695c27b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:46.550299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2f6220da-cbe2-4231-aad7-8f1a5695c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:46.554511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2a264df5-ed6b-45c0-8fab-a4c05ada4786 None None] MFA Rules not processed for user `8010953562b04d11a3da3ccaaf72714b`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:46.555690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2a264df5-ed6b-45c0-8fab-a4c05ada4786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:46.614656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1040/2083] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:46.627249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6220da-cbe2-4231-aad7-8f1a5695c27b None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.628609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2f6220da-cbe2-4231-aad7-8f1a5695c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:46.634474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1044/2084] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:46.646697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a264df5-ed6b-45c0-8fab-a4c05ada4786 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.648690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2a264df5-ed6b-45c0-8fab-a4c05ada4786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:46.712246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6220da-cbe2-4231-aad7-8f1a5695c27b None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.718241 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f3436-52d8-4725-9d59-b6fed239eaaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.718861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f3436-52d8-4725-9d59-b6fed239eaaf service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.719201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f3436-52d8-4725-9d59-b6fed239eaaf service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.719530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f3436-52d8-4725-9d59-b6fed239eaaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.730466 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f3436-52d8-4725-9d59-b6fed239eaaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.732075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f3436-52d8-4725-9d59-b6fed239eaaf service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.732910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a264df5-ed6b-45c0-8fab-a4c05ada4786 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.737346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef4f62-c9a3-4275-af88-acb21a796059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.738610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef4f62-c9a3-4275-af88-acb21a796059 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.738899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef4f62-c9a3-4275-af88-acb21a796059 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.739504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef4f62-c9a3-4275-af88-acb21a796059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.743526 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1041/2085] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:53:46.750037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef4f62-c9a3-4275-af88-acb21a796059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.751484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef4f62-c9a3-4275-af88-acb21a796059 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.761352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1045/2086] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:46.791045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f3436-52d8-4725-9d59-b6fed239eaaf service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:46.801159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f3436-52d8-4725-9d59-b6fed239eaaf service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:46.805938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:46.806578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.806862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.807169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] PATH_INFO: `/v3/users/b70bc2ca226147f8b4fb2a709cfe70e0` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.808503 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b70bc2ca226147f8b4fb2a709cfe70e0)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:46.810471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:46.867496 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b70bc2ca226147f8b4fb2a709cfe70e0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:46.868437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b70bc2ca226147f8b4fb2a709cfe70e0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:46.876245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b70bc2ca226147f8b4fb2a709cfe70e0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:46.884230 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b70bc2ca226147f8b4fb2a709cfe70e0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:46.884987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b70bc2ca226147f8b4fb2a709cfe70e0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:46.905711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1042/2087] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:46 2022] DELETE /identity/v3/users/b70bc2ca226147f8b4fb2a709cfe70e0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:46.910680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9408d5-c2bd-4ea5-aaea-dc8217b8e072 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:46.911160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9408d5-c2bd-4ea5-aaea-dc8217b8e072 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:46.911243 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9408d5-c2bd-4ea5-aaea-dc8217b8e072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:46.931418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-9f9408d5-c2bd-4ea5-aaea-dc8217b8e072 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:46.932681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9f9408d5-c2bd-4ea5-aaea-dc8217b8e072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:46.939861 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:47.013078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1046/2088] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:53:47.023707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:47.025403 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:47.047732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:47.088081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37bb16-33ae-4330-9ec5-f7dbe1886c82 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:47.091967 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14df1a3-2352-410c-8fa0-55ee23f4365c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:47.092575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14df1a3-2352-410c-8fa0-55ee23f4365c service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:47.092852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14df1a3-2352-410c-8fa0-55ee23f4365c service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:47.093676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14df1a3-2352-410c-8fa0-55ee23f4365c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:47.105108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14df1a3-2352-410c-8fa0-55ee23f4365c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:47.106607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14df1a3-2352-410c-8fa0-55ee23f4365c service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:47.118034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1043/2089] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:47.596745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9408d5-c2bd-4ea5-aaea-dc8217b8e072 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:47.604623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9f9408d5-c2bd-4ea5-aaea-dc8217b8e072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:47.629029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9408d5-c2bd-4ea5-aaea-dc8217b8e072 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:47.635223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:47.636021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:47.636736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:47.637084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] PATH_INFO: `/v3/projects/eb178f7036624ee5acb4edeaa8c5192a` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:47.638587 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb178f7036624ee5acb4edeaa8c5192a)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:47.641250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:47.698518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb178f7036624ee5acb4edeaa8c5192a', 'request_id': 'req-b1cd92fd-b6e4-4009-86ea-d7c9613be890'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:47.708258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb178f7036624ee5acb4edeaa8c5192a', 'request_id': 'req-b1cd92fd-b6e4-4009-86ea-d7c9613be890'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:47.717558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb178f7036624ee5acb4edeaa8c5192a', 'request_id': 'req-b1cd92fd-b6e4-4009-86ea-d7c9613be890'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:47.730366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] The token cache is being invalidate because project eb178f7036624ee5acb4edeaa8c5192a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:47.730763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:47.731485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:47.733253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1047/2090] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:47 2022] DELETE /identity/v3/projects/eb178f7036624ee5acb4edeaa8c5192a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:47.737553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14df1a3-2352-410c-8fa0-55ee23f4365c service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:47.754387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14df1a3-2352-410c-8fa0-55ee23f4365c service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:47.838078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:47.840639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:47.841156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:47.841898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] PATH_INFO: `/v3/users/8010953562b04d11a3da3ccaaf72714b` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:47.844402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8010953562b04d11a3da3ccaaf72714b)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:47.847140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:47.915454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8010953562b04d11a3da3ccaaf72714b'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:47.916869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8010953562b04d11a3da3ccaaf72714b'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:47.924801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8010953562b04d11a3da3ccaaf72714b'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:47.934697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8010953562b04d11a3da3ccaaf72714b'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:47.935557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9fab5ba1-4cd9-49ac-bc3c-8ad727a2e7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8010953562b04d11a3da3ccaaf72714b'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:47.957279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1044/2091] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:53:47 2022] DELETE /identity/v3/users/8010953562b04d11a3da3ccaaf72714b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:48.358695 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:48.360302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:48.379262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd92fd-b6e4-4009-86ea-d7c9613be890 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:48.465317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:48.466067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:48.466359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:48.467105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] PATH_INFO: `/v3/projects/6f3412747e564785a902780e85555db4` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:48.470913 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3412747e564785a902780e85555db4)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:48.475037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:48.534772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:48.548015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3412747e564785a902780e85555db4', 'request_id': 'req-7e174edc-8bd3-48a1-9758-855393e67cde'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:48.559369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3412747e564785a902780e85555db4', 'request_id': 'req-7e174edc-8bd3-48a1-9758-855393e67cde'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:48.569916 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3412747e564785a902780e85555db4', 'request_id': 'req-7e174edc-8bd3-48a1-9758-855393e67cde'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:48.582783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] The token cache is being invalidate because project 6f3412747e564785a902780e85555db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:53:48.583163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:48.583888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-7e174edc-8bd3-48a1-9758-855393e67cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:53:48.585471 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1048/2092] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:53:48 2022] DELETE /identity/v3/projects/6f3412747e564785a902780e85555db4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:48.591036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5543c5-284a-42cc-961d-00c78f7796f2 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:48.591320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5543c5-284a-42cc-961d-00c78f7796f2 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:48.591872 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5543c5-284a-42cc-961d-00c78f7796f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:48.613630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-7a5543c5-284a-42cc-961d-00c78f7796f2 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:48.614928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7a5543c5-284a-42cc-961d-00c78f7796f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:48.659850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:48.724716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1045/2093] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:48.733282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597eebb-ece2-49d2-91e7-8ed15c57b2ce None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:48.734081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597eebb-ece2-49d2-91e7-8ed15c57b2ce None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:48.734440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597eebb-ece2-49d2-91e7-8ed15c57b2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:48.758598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-3597eebb-ece2-49d2-91e7-8ed15c57b2ce None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:48.759866 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3597eebb-ece2-49d2-91e7-8ed15c57b2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:48.769542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1049/2094] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:48.772861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5543c5-284a-42cc-961d-00c78f7796f2 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:48.783998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5543c5-284a-42cc-961d-00c78f7796f2 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:48.789010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047b6f1-a178-4aee-9754-54cbce39632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:48.789751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047b6f1-a178-4aee-9754-54cbce39632d admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:48.790104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047b6f1-a178-4aee-9754-54cbce39632d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:48.790446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047b6f1-a178-4aee-9754-54cbce39632d admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:48.791186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047b6f1-a178-4aee-9754-54cbce39632d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:53:48.791558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047b6f1-a178-4aee-9754-54cbce39632d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:48.793299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047b6f1-a178-4aee-9754-54cbce39632d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:48.808013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1046/2095] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:53:48.812324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597eebb-ece2-49d2-91e7-8ed15c57b2ce None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:48.823028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597eebb-ece2-49d2-91e7-8ed15c57b2ce None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:48.828582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aa8d7-58b1-46ca-a356-c21dc1e7a2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:48.829117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aa8d7-58b1-46ca-a356-c21dc1e7a2be admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:48.829603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aa8d7-58b1-46ca-a356-c21dc1e7a2be admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:48.829783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aa8d7-58b1-46ca-a356-c21dc1e7a2be admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:48.830432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aa8d7-58b1-46ca-a356-c21dc1e7a2be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:48.832393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aa8d7-58b1-46ca-a356-c21dc1e7a2be admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:48.872832 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1050/2096] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:48 2022] POST /identity/v3/projects => generated 407 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:48.877268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047b6f1-a178-4aee-9754-54cbce39632d admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:48.887684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047b6f1-a178-4aee-9754-54cbce39632d admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:48.948880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce21697-8314-4b1e-b034-a66e9096914e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:48.949414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21697-8314-4b1e-b034-a66e9096914e admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:48.949724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21697-8314-4b1e-b034-a66e9096914e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:48.949995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21697-8314-4b1e-b034-a66e9096914e admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:48.950638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce21697-8314-4b1e-b034-a66e9096914e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:48.953206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce21697-8314-4b1e-b034-a66e9096914e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:48.976010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1047/2097] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:48 2022] POST /identity/v3/users => generated 304 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:48.980220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aa8d7-58b1-46ca-a356-c21dc1e7a2be admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:48.990503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aa8d7-58b1-46ca-a356-c21dc1e7a2be admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:48.995256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c02d2-2676-4a10-8397-b2079ed15ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:48.995984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c02d2-2676-4a10-8397-b2079ed15ffd admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:48.996336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c02d2-2676-4a10-8397-b2079ed15ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:48.996672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c02d2-2676-4a10-8397-b2079ed15ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:48.997390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c02d2-2676-4a10-8397-b2079ed15ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:48.999123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c02d2-2676-4a10-8397-b2079ed15ffd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:49.009571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1051/2098] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:49.013931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce21697-8314-4b1e-b034-a66e9096914e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:49.026270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce21697-8314-4b1e-b034-a66e9096914e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:49.031642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d08826-b2e6-4d8e-8177-1aba64ff9f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:49.032252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d08826-b2e6-4d8e-8177-1aba64ff9f97 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:49.032533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d08826-b2e6-4d8e-8177-1aba64ff9f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:49.032806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d08826-b2e6-4d8e-8177-1aba64ff9f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:49.033479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d08826-b2e6-4d8e-8177-1aba64ff9f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:49.035108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d08826-b2e6-4d8e-8177-1aba64ff9f97 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:49.046530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1048/2099] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:49.050717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c02d2-2676-4a10-8397-b2079ed15ffd admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:49.060812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c02d2-2676-4a10-8397-b2079ed15ffd admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:49.065434 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48fe93-5ad8-4e63-90d0-b7485b5f52e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:49.066117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48fe93-5ad8-4e63-90d0-b7485b5f52e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:49.066396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48fe93-5ad8-4e63-90d0-b7485b5f52e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:49.066669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48fe93-5ad8-4e63-90d0-b7485b5f52e8 admin admin] PATH_INFO: `/v3/projects/3c76066cf3484bbaa1a90f998e586fbb/users/863a11b8580448e281472e9ac2db22b7/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:49.082144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48fe93-5ad8-4e63-90d0-b7485b5f52e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c76066cf3484bbaa1a90f998e586fbb, user_id=863a11b8580448e281472e9ac2db22b7, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:49.084791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48fe93-5ad8-4e63-90d0-b7485b5f52e8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:49.099805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1052/2100] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:49 2022] PUT /identity/v3/projects/3c76066cf3484bbaa1a90f998e586fbb/users/863a11b8580448e281472e9ac2db22b7/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:49.112460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d568f9-3dfc-4186-9ca3-657b61455886 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:49.112832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d568f9-3dfc-4186-9ca3-657b61455886 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:49.113238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d568f9-3dfc-4186-9ca3-657b61455886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:49.144581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-52d568f9-3dfc-4186-9ca3-657b61455886 None None] MFA Rules not processed for user `863a11b8580448e281472e9ac2db22b7`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:49.148123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-52d568f9-3dfc-4186-9ca3-657b61455886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:49.263160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1049/2101] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:49 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:49.268340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48fe93-5ad8-4e63-90d0-b7485b5f52e8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:49.370399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48fe93-5ad8-4e63-90d0-b7485b5f52e8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:49.375841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46146b17-773c-4567-873a-32efd7414721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:49.376464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46146b17-773c-4567-873a-32efd7414721 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:49.376738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46146b17-773c-4567-873a-32efd7414721 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:49.377192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46146b17-773c-4567-873a-32efd7414721 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:49.377874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46146b17-773c-4567-873a-32efd7414721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:49.379921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46146b17-773c-4567-873a-32efd7414721 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:49.412403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1053/2102] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:49 2022] POST /identity/v3/projects => generated 409 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:49.416117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d568f9-3dfc-4186-9ca3-657b61455886 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:49.429969 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d568f9-3dfc-4186-9ca3-657b61455886 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:49.465524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:49.497650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41833-2c81-456b-9cc9-570b2678b903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:49.498274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41833-2c81-456b-9cc9-570b2678b903 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:49.498554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41833-2c81-456b-9cc9-570b2678b903 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:49.500111 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41833-2c81-456b-9cc9-570b2678b903 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:49.500786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a41833-2c81-456b-9cc9-570b2678b903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:49.504090 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a41833-2c81-456b-9cc9-570b2678b903 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:49.533539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1050/2103] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:49 2022] POST /identity/v3/users => generated 311 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:53:49.537020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46146b17-773c-4567-873a-32efd7414721 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:49.549780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46146b17-773c-4567-873a-32efd7414721 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:49.555436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a9552-cfc3-4d1b-8733-0efca09ea5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:49.556239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a9552-cfc3-4d1b-8733-0efca09ea5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:49.556662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a9552-cfc3-4d1b-8733-0efca09ea5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:49.557082 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a9552-cfc3-4d1b-8733-0efca09ea5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:49.557992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a9552-cfc3-4d1b-8733-0efca09ea5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:49.561073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a9552-cfc3-4d1b-8733-0efca09ea5f8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:49.573574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1054/2104] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:49.578563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41833-2c81-456b-9cc9-570b2678b903 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:49.589807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41833-2c81-456b-9cc9-570b2678b903 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:49.595236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742e39f-1cf2-4009-a988-2d71be708129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:49.595958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742e39f-1cf2-4009-a988-2d71be708129 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:49.596302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742e39f-1cf2-4009-a988-2d71be708129 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:49.596640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742e39f-1cf2-4009-a988-2d71be708129 admin admin] PATH_INFO: `/v3/projects/e850065efdac476aa322a591089a002b/users/524f0a850dc04b60abb1fa5ea95b4b65/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:49.615205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742e39f-1cf2-4009-a988-2d71be708129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e850065efdac476aa322a591089a002b, user_id=524f0a850dc04b60abb1fa5ea95b4b65, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:49.617983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742e39f-1cf2-4009-a988-2d71be708129 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:49.633549 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1051/2105] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:49 2022] PUT /identity/v3/projects/e850065efdac476aa322a591089a002b/users/524f0a850dc04b60abb1fa5ea95b4b65/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:53:49.636818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a9552-cfc3-4d1b-8733-0efca09ea5f8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:49.647374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a9552-cfc3-4d1b-8733-0efca09ea5f8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:49.687567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:49.718023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b5f88-aa33-47f1-b5e6-2bd03f67bdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:49.718673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b5f88-aa33-47f1-b5e6-2bd03f67bdfd admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:49.719579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b5f88-aa33-47f1-b5e6-2bd03f67bdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:49.719579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b5f88-aa33-47f1-b5e6-2bd03f67bdfd admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:49.720081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b5f88-aa33-47f1-b5e6-2bd03f67bdfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:49.721890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b5f88-aa33-47f1-b5e6-2bd03f67bdfd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:49.733918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1055/2106] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:53:49.737084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742e39f-1cf2-4009-a988-2d71be708129 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:49.748368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742e39f-1cf2-4009-a988-2d71be708129 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:49.755770 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07e8db-c304-428f-bb62-f18cd05c3e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:49.757007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07e8db-c304-428f-bb62-f18cd05c3e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:49.757298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07e8db-c304-428f-bb62-f18cd05c3e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:49.757629 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07e8db-c304-428f-bb62-f18cd05c3e61 admin admin] PATH_INFO: `/v3/projects/e850065efdac476aa322a591089a002b/users/524f0a850dc04b60abb1fa5ea95b4b65/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:49.759938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07e8db-c304-428f-bb62-f18cd05c3e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e850065efdac476aa322a591089a002b, user_id=524f0a850dc04b60abb1fa5ea95b4b65, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:49.766537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07e8db-c304-428f-bb62-f18cd05c3e61 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:49.793089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18eb72-9ead-4feb-b300-74728fbc68e8 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:49.793429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18eb72-9ead-4feb-b300-74728fbc68e8 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:49.793678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18eb72-9ead-4feb-b300-74728fbc68e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:49.799131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1052/2107] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:49 2022] PUT /identity/v3/projects/e850065efdac476aa322a591089a002b/users/524f0a850dc04b60abb1fa5ea95b4b65/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:49.842777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2b18eb72-9ead-4feb-b300-74728fbc68e8 None None] MFA Rules not processed for user `524f0a850dc04b60abb1fa5ea95b4b65`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:49.846541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2b18eb72-9ead-4feb-b300-74728fbc68e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:49.949283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07e8db-c304-428f-bb62-f18cd05c3e61 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:49.952390 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7a07e8db-c304-428f-bb62-f18cd05c3e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:49.956606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1056/2108] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:49 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:50.031276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07e8db-c304-428f-bb62-f18cd05c3e61 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:50.035493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301c9a4-2d97-458e-90fc-8efb0f754645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:50.036333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301c9a4-2d97-458e-90fc-8efb0f754645 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:50.036748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301c9a4-2d97-458e-90fc-8efb0f754645 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:50.037173 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301c9a4-2d97-458e-90fc-8efb0f754645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:50.050005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301c9a4-2d97-458e-90fc-8efb0f754645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:50.051788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301c9a4-2d97-458e-90fc-8efb0f754645 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:50.064339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1053/2109] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:49 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:53:50.306866 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18eb72-9ead-4feb-b300-74728fbc68e8 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:50.308706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2b18eb72-9ead-4feb-b300-74728fbc68e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:50.330419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18eb72-9ead-4feb-b300-74728fbc68e8 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:50.334706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508268b-f494-416a-a9f2-cc852e6fa17c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:50.335317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508268b-f494-416a-a9f2-cc852e6fa17c service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:50.335670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508268b-f494-416a-a9f2-cc852e6fa17c service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:50.335948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508268b-f494-416a-a9f2-cc852e6fa17c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:50.433360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508268b-f494-416a-a9f2-cc852e6fa17c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:50.435925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508268b-f494-416a-a9f2-cc852e6fa17c service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:50.449741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1057/2110] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:50 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:53:51.855235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6b4c1-b960-4d0a-b6ee-6844cffa3078 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:51.856108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6b4c1-b960-4d0a-b6ee-6844cffa3078 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:51.856543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6b4c1-b960-4d0a-b6ee-6844cffa3078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:51.892659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-9db6b4c1-b960-4d0a-b6ee-6844cffa3078 None None] MFA Rules not processed for user `7cfae014351641d8a949775875a45958`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:51.894124 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9db6b4c1-b960-4d0a-b6ee-6844cffa3078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:51.985077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1054/2111] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:51.989991 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508268b-f494-416a-a9f2-cc852e6fa17c service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:51.991775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b508268b-f494-416a-a9f2-cc852e6fa17c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:52.073750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:52.080560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508268b-f494-416a-a9f2-cc852e6fa17c service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:52.085196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbacaf9-5d2c-49c2-83c8-ee5120c97397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:52.085784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbacaf9-5d2c-49c2-83c8-ee5120c97397 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:52.086068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbacaf9-5d2c-49c2-83c8-ee5120c97397 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:52.086342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbacaf9-5d2c-49c2-83c8-ee5120c97397 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:52.086994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbacaf9-5d2c-49c2-83c8-ee5120c97397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:52.088914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbacaf9-5d2c-49c2-83c8-ee5120c97397 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:52.120589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1058/2112] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:53:51 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:52.124913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6b4c1-b960-4d0a-b6ee-6844cffa3078 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:52.137120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6b4c1-b960-4d0a-b6ee-6844cffa3078 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:52.150892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:53:52.204333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dd5df-97b8-4e0b-8b55-370508e3710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:52.205071 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dd5df-97b8-4e0b-8b55-370508e3710c admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:52.205451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dd5df-97b8-4e0b-8b55-370508e3710c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:52.205851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dd5df-97b8-4e0b-8b55-370508e3710c admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:52.206771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311dd5df-97b8-4e0b-8b55-370508e3710c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:52.209194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311dd5df-97b8-4e0b-8b55-370508e3710c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:52.233028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1055/2113] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:53:52 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:53:52.236949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbacaf9-5d2c-49c2-83c8-ee5120c97397 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:52.282818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbacaf9-5d2c-49c2-83c8-ee5120c97397 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:52.288187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10948005-54b9-4d05-8519-50c34d0af433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:52.288942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10948005-54b9-4d05-8519-50c34d0af433 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:52.289344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10948005-54b9-4d05-8519-50c34d0af433 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:52.289710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10948005-54b9-4d05-8519-50c34d0af433 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:52.290441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10948005-54b9-4d05-8519-50c34d0af433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:52.291997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10948005-54b9-4d05-8519-50c34d0af433 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:52.314081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1059/2114] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:53:52.316186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dd5df-97b8-4e0b-8b55-370508e3710c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:52.333576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dd5df-97b8-4e0b-8b55-370508e3710c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:52.339816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522d64d-c359-409f-b211-9331d8ad6bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:52.340646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522d64d-c359-409f-b211-9331d8ad6bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:52.340921 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522d64d-c359-409f-b211-9331d8ad6bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:52.341199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522d64d-c359-409f-b211-9331d8ad6bbf admin admin] PATH_INFO: `/v3/projects/b761452ca4a9445f802ec1f2e22513c4/users/c480a3ee842149a89c5309598fea23d3/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:52.358083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522d64d-c359-409f-b211-9331d8ad6bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b761452ca4a9445f802ec1f2e22513c4, user_id=c480a3ee842149a89c5309598fea23d3, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:52.360984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522d64d-c359-409f-b211-9331d8ad6bbf admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:52.378859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1056/2115] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:52 2022] PUT /identity/v3/projects/b761452ca4a9445f802ec1f2e22513c4/users/c480a3ee842149a89c5309598fea23d3/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:52.382614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10948005-54b9-4d05-8519-50c34d0af433 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:52.393882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10948005-54b9-4d05-8519-50c34d0af433 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:52.480675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bcc2d-69f6-47b1-9346-13c1d4410846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:52.481416 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcc2d-69f6-47b1-9346-13c1d4410846 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:52.481777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcc2d-69f6-47b1-9346-13c1d4410846 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:52.482114 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcc2d-69f6-47b1-9346-13c1d4410846 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:52.482835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bcc2d-69f6-47b1-9346-13c1d4410846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:52.484553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bcc2d-69f6-47b1-9346-13c1d4410846 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:52.495864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1060/2116] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:53:52.500476 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522d64d-c359-409f-b211-9331d8ad6bbf admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:52.511096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522d64d-c359-409f-b211-9331d8ad6bbf admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:52.517245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1dd2e-1f65-4f05-b88d-eae70f27d19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:52.517935 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1dd2e-1f65-4f05-b88d-eae70f27d19c admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:52.518238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1dd2e-1f65-4f05-b88d-eae70f27d19c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:52.518507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1dd2e-1f65-4f05-b88d-eae70f27d19c admin admin] PATH_INFO: `/v3/projects/b761452ca4a9445f802ec1f2e22513c4/users/c480a3ee842149a89c5309598fea23d3/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:52.520417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1dd2e-1f65-4f05-b88d-eae70f27d19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b761452ca4a9445f802ec1f2e22513c4, user_id=c480a3ee842149a89c5309598fea23d3, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:52.523138 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1dd2e-1f65-4f05-b88d-eae70f27d19c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:52.534221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1057/2117] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:53:52 2022] PUT /identity/v3/projects/b761452ca4a9445f802ec1f2e22513c4/users/c480a3ee842149a89c5309598fea23d3/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:53:56.892185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673c7b-146b-4e04-9a3e-755b49ca62f1 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:56.892757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673c7b-146b-4e04-9a3e-755b49ca62f1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:56.892992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673c7b-146b-4e04-9a3e-755b49ca62f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:56.923745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-35673c7b-146b-4e04-9a3e-755b49ca62f1 None None] MFA Rules not processed for user `c480a3ee842149a89c5309598fea23d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:53:56.925001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-35673c7b-146b-4e04-9a3e-755b49ca62f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:56.995553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1061/2118] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:53:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:53:57.009483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1dd2e-1f65-4f05-b88d-eae70f27d19c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:53:57.011337 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-54f1dd2e-1f65-4f05-b88d-eae70f27d19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:53:57.094056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1dd2e-1f65-4f05-b88d-eae70f27d19c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:53:57.100601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59e539-5939-4617-8e48-dd0bb6a89baf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:53:57.101310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59e539-5939-4617-8e48-dd0bb6a89baf service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:53:57.101631 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59e539-5939-4617-8e48-dd0bb6a89baf service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:53:57.101905 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59e539-5939-4617-8e48-dd0bb6a89baf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:53:57.169343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59e539-5939-4617-8e48-dd0bb6a89baf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:53:57.171032 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59e539-5939-4617-8e48-dd0bb6a89baf service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:53:57.180566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1058/2119] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:53:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:54:02.500154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35673c7b-146b-4e04-9a3e-755b49ca62f1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:02.552087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:02.565826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35673c7b-146b-4e04-9a3e-755b49ca62f1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:02.570574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:02.571225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:02.571510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:02.571811 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] PATH_INFO: `/v3/users/863a11b8580448e281472e9ac2db22b7` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:02.573029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=863a11b8580448e281472e9ac2db22b7)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:02.575097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:02.622549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '863a11b8580448e281472e9ac2db22b7'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:02.623209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '863a11b8580448e281472e9ac2db22b7'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:02.629965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '863a11b8580448e281472e9ac2db22b7'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:02.642366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '863a11b8580448e281472e9ac2db22b7'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:02.650073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '863a11b8580448e281472e9ac2db22b7'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:02.656788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1062/2120] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:54:02 2022] DELETE /identity/v3/users/863a11b8580448e281472e9ac2db22b7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:02.661244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b3a09-a597-4e72-8919-3b79d6341595 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:02.661747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b3a09-a597-4e72-8919-3b79d6341595 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:02.661865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b3a09-a597-4e72-8919-3b79d6341595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:02.677342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:02.683953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-bd4b3a09-a597-4e72-8919-3b79d6341595 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:02.685169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bd4b3a09-a597-4e72-8919-3b79d6341595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:02.743962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1059/2121] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:02.754068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:02.755471 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:02.812696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c6544-6b69-4a1f-8159-2e09691d9899 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:02.816338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e1239-bec7-4f1b-91a5-32cd89b7007a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:02.816941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e1239-bec7-4f1b-91a5-32cd89b7007a service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:02.817220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e1239-bec7-4f1b-91a5-32cd89b7007a service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:02.817489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e1239-bec7-4f1b-91a5-32cd89b7007a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:02.826272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e1239-bec7-4f1b-91a5-32cd89b7007a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:02.828369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e1239-bec7-4f1b-91a5-32cd89b7007a service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:02.837778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1063/2122] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:54:03.114587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b3a09-a597-4e72-8919-3b79d6341595 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:03.116099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bd4b3a09-a597-4e72-8919-3b79d6341595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:03.137354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b3a09-a597-4e72-8919-3b79d6341595 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:03.151764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:03.152508 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:03.152789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:03.153060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] PATH_INFO: `/v3/projects/3c76066cf3484bbaa1a90f998e586fbb` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:03.155289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c76066cf3484bbaa1a90f998e586fbb)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:03.157202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:03.209983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c76066cf3484bbaa1a90f998e586fbb', 'request_id': 'req-a20f96b1-6f8f-445c-b36d-61302765c244'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.218070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c76066cf3484bbaa1a90f998e586fbb', 'request_id': 'req-a20f96b1-6f8f-445c-b36d-61302765c244'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.224290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c76066cf3484bbaa1a90f998e586fbb', 'request_id': 'req-a20f96b1-6f8f-445c-b36d-61302765c244'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.235066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] The token cache is being invalidate because project 3c76066cf3484bbaa1a90f998e586fbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:54:03.235368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.236022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.237580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1060/2123] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:54:03 2022] DELETE /identity/v3/projects/3c76066cf3484bbaa1a90f998e586fbb => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:03.241743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e1239-bec7-4f1b-91a5-32cd89b7007a service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:03.251242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e1239-bec7-4f1b-91a5-32cd89b7007a service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:03.310727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:03.311360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:03.311636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:03.311906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] PATH_INFO: `/v3/users/524f0a850dc04b60abb1fa5ea95b4b65` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:03.313024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=524f0a850dc04b60abb1fa5ea95b4b65)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:03.314962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:03.328600 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:03.364064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '524f0a850dc04b60abb1fa5ea95b4b65'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.364702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '524f0a850dc04b60abb1fa5ea95b4b65'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.371411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '524f0a850dc04b60abb1fa5ea95b4b65'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.383152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '524f0a850dc04b60abb1fa5ea95b4b65'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.390737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-60f2db4c-dd60-46df-98b3-4bc1b5b4bf4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '524f0a850dc04b60abb1fa5ea95b4b65'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.396588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1064/2124] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:54:03 2022] DELETE /identity/v3/users/524f0a850dc04b60abb1fa5ea95b4b65 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:03.684359 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:03.686644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:03.699760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f96b1-6f8f-445c-b36d-61302765c244 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:03.759456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:03.760159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:03.760357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:03.760630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] PATH_INFO: `/v3/projects/e850065efdac476aa322a591089a002b` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:03.761794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e850065efdac476aa322a591089a002b)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:03.763698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:03.809620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:03.826145 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e850065efdac476aa322a591089a002b', 'request_id': 'req-10b659a8-5f52-40a7-9c24-70ae3153eb17'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.835126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e850065efdac476aa322a591089a002b', 'request_id': 'req-10b659a8-5f52-40a7-9c24-70ae3153eb17'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.841945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e850065efdac476aa322a591089a002b', 'request_id': 'req-10b659a8-5f52-40a7-9c24-70ae3153eb17'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.853762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] The token cache is being invalidate because project e850065efdac476aa322a591089a002b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:54:03.854049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.854682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b659a8-5f52-40a7-9c24-70ae3153eb17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:03.856198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1061/2125] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:54:03 2022] DELETE /identity/v3/projects/e850065efdac476aa322a591089a002b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:03.861695 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdb822-cd91-496c-969f-1fdd115f137b None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:03.862147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdb822-cd91-496c-969f-1fdd115f137b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:03.862362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdb822-cd91-496c-969f-1fdd115f137b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:03.882845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-38cdb822-cd91-496c-969f-1fdd115f137b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:03.884095 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-38cdb822-cd91-496c-969f-1fdd115f137b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:03.950302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1065/2126] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:54:03.958838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda0ec0-babf-49a0-83d2-9be5f00fbf0d None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:03.959277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda0ec0-babf-49a0-83d2-9be5f00fbf0d None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:03.959667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda0ec0-babf-49a0-83d2-9be5f00fbf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:03.980501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-3cda0ec0-babf-49a0-83d2-9be5f00fbf0d None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:03.981737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3cda0ec0-babf-49a0-83d2-9be5f00fbf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:03.989443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1062/2127] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:03.993753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdb822-cd91-496c-969f-1fdd115f137b None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:04.003594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdb822-cd91-496c-969f-1fdd115f137b None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:04.008228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbb154-2d82-4a76-a51b-2caec72ebd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:04.008803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbb154-2d82-4a76-a51b-2caec72ebd86 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:04.009083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbb154-2d82-4a76-a51b-2caec72ebd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:04.009354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbb154-2d82-4a76-a51b-2caec72ebd86 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:04.010051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbb154-2d82-4a76-a51b-2caec72ebd86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:54:04.010358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbb154-2d82-4a76-a51b-2caec72ebd86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:04.011961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbb154-2d82-4a76-a51b-2caec72ebd86 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:04.025108 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1066/2128] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:54:04.029309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda0ec0-babf-49a0-83d2-9be5f00fbf0d None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:04.039092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda0ec0-babf-49a0-83d2-9be5f00fbf0d None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:04.043651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b143c-0bae-40f5-9b3a-4a878941ec44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:04.044234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b143c-0bae-40f5-9b3a-4a878941ec44 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:04.044515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b143c-0bae-40f5-9b3a-4a878941ec44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:04.044771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b143c-0bae-40f5-9b3a-4a878941ec44 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:04.045421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b143c-0bae-40f5-9b3a-4a878941ec44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:04.047329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b143c-0bae-40f5-9b3a-4a878941ec44 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:04.074248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1063/2129] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:54:04 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:04.078956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbb154-2d82-4a76-a51b-2caec72ebd86 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:04.089065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbb154-2d82-4a76-a51b-2caec72ebd86 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:04.153372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6ef5a-22e6-48b4-9ce5-89ad7a4d3888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:04.154019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6ef5a-22e6-48b4-9ce5-89ad7a4d3888 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:04.154305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6ef5a-22e6-48b4-9ce5-89ad7a4d3888 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:04.154615 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6ef5a-22e6-48b4-9ce5-89ad7a4d3888 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:04.155249 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6ef5a-22e6-48b4-9ce5-89ad7a4d3888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:04.157191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6ef5a-22e6-48b4-9ce5-89ad7a4d3888 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:04.180632 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1067/2130] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:54:04 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:04.181804 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b143c-0bae-40f5-9b3a-4a878941ec44 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:04.192844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b143c-0bae-40f5-9b3a-4a878941ec44 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:04.197937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c739d-de87-4b1b-a951-79331b764898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:04.198651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c739d-de87-4b1b-a951-79331b764898 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:04.198935 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c739d-de87-4b1b-a951-79331b764898 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:04.199214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c739d-de87-4b1b-a951-79331b764898 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:04.199949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c739d-de87-4b1b-a951-79331b764898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:04.201597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c739d-de87-4b1b-a951-79331b764898 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:04.212498 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1064/2131] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:04.216861 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6ef5a-22e6-48b4-9ce5-89ad7a4d3888 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:04.226983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6ef5a-22e6-48b4-9ce5-89ad7a4d3888 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:04.232464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91745f8b-ed32-425a-8ff5-55507cecfe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:04.233595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91745f8b-ed32-425a-8ff5-55507cecfe04 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:04.233875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91745f8b-ed32-425a-8ff5-55507cecfe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:04.234146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91745f8b-ed32-425a-8ff5-55507cecfe04 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:04.235171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91745f8b-ed32-425a-8ff5-55507cecfe04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:04.236958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91745f8b-ed32-425a-8ff5-55507cecfe04 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:04.240057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:04.270304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1068/2132] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:04.276512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c739d-de87-4b1b-a951-79331b764898 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:04.287408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c739d-de87-4b1b-a951-79331b764898 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:04.292044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19d5f5-f936-4f26-8b75-85a5969c4a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:04.292699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19d5f5-f936-4f26-8b75-85a5969c4a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:04.292981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19d5f5-f936-4f26-8b75-85a5969c4a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:04.293384 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19d5f5-f936-4f26-8b75-85a5969c4a04 admin admin] PATH_INFO: `/v3/projects/f1d546ddcaca4842a7c598b33ff06ff5/users/0bdbf201476a437eac34a809e137c379/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:04.310530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19d5f5-f936-4f26-8b75-85a5969c4a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d546ddcaca4842a7c598b33ff06ff5, user_id=0bdbf201476a437eac34a809e137c379, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:04.313258 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19d5f5-f936-4f26-8b75-85a5969c4a04 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:04.327310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1065/2133] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:54:04 2022] PUT /identity/v3/projects/f1d546ddcaca4842a7c598b33ff06ff5/users/0bdbf201476a437eac34a809e137c379/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:04.332627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43423b-1abc-4b7c-bf06-0719e0e1e161 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:04.332925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43423b-1abc-4b7c-bf06-0719e0e1e161 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:04.333177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43423b-1abc-4b7c-bf06-0719e0e1e161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:04.354621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-4f43423b-1abc-4b7c-bf06-0719e0e1e161 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:04.356406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4f43423b-1abc-4b7c-bf06-0719e0e1e161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:04.418136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1069/2134] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:04.428193 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19d5f5-f936-4f26-8b75-85a5969c4a04 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:04.429929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7a19d5f5-f936-4f26-8b75-85a5969c4a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:04.485755 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19d5f5-f936-4f26-8b75-85a5969c4a04 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:04.489264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a74024-10f5-461b-8448-f563ea328eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:04.489879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74024-10f5-461b-8448-f563ea328eed service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:04.490155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74024-10f5-461b-8448-f563ea328eed service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:04.490428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74024-10f5-461b-8448-f563ea328eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:04.500072 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a74024-10f5-461b-8448-f563ea328eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:04.501726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a74024-10f5-461b-8448-f563ea328eed service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:04.512040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1066/2135] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:54:05.155905 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6f46e-0d7b-4ef1-80a2-6219b0918b3c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:05.156467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6f46e-0d7b-4ef1-80a2-6219b0918b3c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:05.156467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6f46e-0d7b-4ef1-80a2-6219b0918b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:05.185236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b4a6f46e-0d7b-4ef1-80a2-6219b0918b3c None None] MFA Rules not processed for user `0bdbf201476a437eac34a809e137c379`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:05.186558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b4a6f46e-0d7b-4ef1-80a2-6219b0918b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:05.257265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1070/2136] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:54:05.261351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a74024-10f5-461b-8448-f563ea328eed service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:05.271472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a74024-10f5-461b-8448-f563ea328eed service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:05.276514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e63a-5710-4c13-b97f-133aa0d54220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:05.277178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e63a-5710-4c13-b97f-133aa0d54220 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:05.277399 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e63a-5710-4c13-b97f-133aa0d54220 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:05.277881 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e63a-5710-4c13-b97f-133aa0d54220 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:05.278536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514e63a-5710-4c13-b97f-133aa0d54220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:05.280974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514e63a-5710-4c13-b97f-133aa0d54220 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:05.315186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1067/2137] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:54:05 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:05.316435 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6f46e-0d7b-4ef1-80a2-6219b0918b3c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:05.328490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6f46e-0d7b-4ef1-80a2-6219b0918b3c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:05.394943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dfafc-7d59-4a3d-8e27-6c4381a5d7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:05.395628 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dfafc-7d59-4a3d-8e27-6c4381a5d7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:05.395915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dfafc-7d59-4a3d-8e27-6c4381a5d7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:05.396188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dfafc-7d59-4a3d-8e27-6c4381a5d7a1 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:05.396851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dfafc-7d59-4a3d-8e27-6c4381a5d7a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:05.398851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dfafc-7d59-4a3d-8e27-6c4381a5d7a1 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:05.421590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1071/2138] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:54:05 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:05.425797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e63a-5710-4c13-b97f-133aa0d54220 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:05.436824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e63a-5710-4c13-b97f-133aa0d54220 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:05.441419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd192e1-7bda-4148-8645-991b553a4a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:05.442047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd192e1-7bda-4148-8645-991b553a4a8d admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:05.442325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd192e1-7bda-4148-8645-991b553a4a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:05.442591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd192e1-7bda-4148-8645-991b553a4a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:05.443225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd192e1-7bda-4148-8645-991b553a4a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:05.444954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd192e1-7bda-4148-8645-991b553a4a8d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:05.457828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1068/2139] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:05.462115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dfafc-7d59-4a3d-8e27-6c4381a5d7a1 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:05.473628 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dfafc-7d59-4a3d-8e27-6c4381a5d7a1 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:05.483016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a1603-7986-4d46-bfda-d8ecfae83dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:05.483703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a1603-7986-4d46-bfda-d8ecfae83dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:05.483987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a1603-7986-4d46-bfda-d8ecfae83dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:05.484265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a1603-7986-4d46-bfda-d8ecfae83dc1 admin admin] PATH_INFO: `/v3/projects/9b28232f58b043199222b1a1b3b589a9/users/e2416ea6b67b49cb915e0b3fca243832/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:05.500505 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a1603-7986-4d46-bfda-d8ecfae83dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b28232f58b043199222b1a1b3b589a9, user_id=e2416ea6b67b49cb915e0b3fca243832, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:05.503351 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a1603-7986-4d46-bfda-d8ecfae83dc1 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:05.505590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:05.518870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1072/2140] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:54:05 2022] PUT /identity/v3/projects/9b28232f58b043199222b1a1b3b589a9/users/e2416ea6b67b49cb915e0b3fca243832/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:05.522936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd192e1-7bda-4148-8645-991b553a4a8d admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:05.533685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd192e1-7bda-4148-8645-991b553a4a8d admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:05.558301 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:05.593593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77974ac7-c44e-4724-91f8-86617d0ea9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:05.594129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77974ac7-c44e-4724-91f8-86617d0ea9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:05.594407 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77974ac7-c44e-4724-91f8-86617d0ea9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:05.594713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77974ac7-c44e-4724-91f8-86617d0ea9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:05.595333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77974ac7-c44e-4724-91f8-86617d0ea9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:05.597012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77974ac7-c44e-4724-91f8-86617d0ea9b9 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:05.609561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1069/2141] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:05.613918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a1603-7986-4d46-bfda-d8ecfae83dc1 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:05.624146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a1603-7986-4d46-bfda-d8ecfae83dc1 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:05.628700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba2dde-5e71-4ef5-87de-cfbbbe0facb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:05.629342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba2dde-5e71-4ef5-87de-cfbbbe0facb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:05.629663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba2dde-5e71-4ef5-87de-cfbbbe0facb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:05.629937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba2dde-5e71-4ef5-87de-cfbbbe0facb6 admin admin] PATH_INFO: `/v3/projects/9b28232f58b043199222b1a1b3b589a9/users/e2416ea6b67b49cb915e0b3fca243832/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:05.631818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba2dde-5e71-4ef5-87de-cfbbbe0facb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b28232f58b043199222b1a1b3b589a9, user_id=e2416ea6b67b49cb915e0b3fca243832, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:05.634501 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba2dde-5e71-4ef5-87de-cfbbbe0facb6 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:05.645098 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1073/2142] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:54:05 2022] PUT /identity/v3/projects/9b28232f58b043199222b1a1b3b589a9/users/e2416ea6b67b49cb915e0b3fca243832/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:06.266368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e140e06-6978-49a3-a02b-ca3e33f1b3fd None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:06.266874 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e140e06-6978-49a3-a02b-ca3e33f1b3fd None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:06.266965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e140e06-6978-49a3-a02b-ca3e33f1b3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:06.307477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-1e140e06-6978-49a3-a02b-ca3e33f1b3fd None None] MFA Rules not processed for user `e2416ea6b67b49cb915e0b3fca243832`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:06.308794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1e140e06-6978-49a3-a02b-ca3e33f1b3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:06.392385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1070/2143] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:54:06.403602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba2dde-5e71-4ef5-87de-cfbbbe0facb6 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:06.405949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a8ba2dde-5e71-4ef5-87de-cfbbbe0facb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:06.468485 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba2dde-5e71-4ef5-87de-cfbbbe0facb6 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:06.472091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5e7cf-ca85-4c5a-b021-f8a80e448cc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:06.472691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5e7cf-ca85-4c5a-b021-f8a80e448cc7 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:06.472966 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5e7cf-ca85-4c5a-b021-f8a80e448cc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:06.473237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5e7cf-ca85-4c5a-b021-f8a80e448cc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:06.530486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5e7cf-ca85-4c5a-b021-f8a80e448cc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:06.531948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5e7cf-ca85-4c5a-b021-f8a80e448cc7 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:06.540607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1074/2144] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:54:08.991519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e140e06-6978-49a3-a02b-ca3e33f1b3fd None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:09.000809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e140e06-6978-49a3-a02b-ca3e33f1b3fd None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:09.004325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3abe06-86de-4560-b322-249a1ffba467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:09.004913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3abe06-86de-4560-b322-249a1ffba467 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:09.005187 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3abe06-86de-4560-b322-249a1ffba467 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:09.005452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3abe06-86de-4560-b322-249a1ffba467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:09.014525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3abe06-86de-4560-b322-249a1ffba467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:09.015991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3abe06-86de-4560-b322-249a1ffba467 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:09.026637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1071/2145] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:54:16.536651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:INSERT=2,keystone:SELECT=55 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:19.021777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=57 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:32.756149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007288a-4335-4948-833d-89dd0f1bb31c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:32.756149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007288a-4335-4948-833d-89dd0f1bb31c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:32.756149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007288a-4335-4948-833d-89dd0f1bb31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:32.776944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-1007288a-4335-4948-833d-89dd0f1bb31c None None] MFA Rules not processed for user `7cfae014351641d8a949775875a45958`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:32.778204 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1007288a-4335-4948-833d-89dd0f1bb31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:32.839120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1075/2146] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:33.509885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfe840-b9ef-48ce-a47f-72abcfa5673e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:33.510433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfe840-b9ef-48ce-a47f-72abcfa5673e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:33.510433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfe840-b9ef-48ce-a47f-72abcfa5673e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:33.529954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-92dfe840-b9ef-48ce-a47f-72abcfa5673e None None] MFA Rules not processed for user `335dd8706d4c402e98605479b57d059a`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:33.531176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-92dfe840-b9ef-48ce-a47f-72abcfa5673e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:33.589394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1072/2147] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:33.695910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007288a-4335-4948-833d-89dd0f1bb31c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:33.697470 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1007288a-4335-4948-833d-89dd0f1bb31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:33.784054 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007288a-4335-4948-833d-89dd0f1bb31c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:33.789178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:33.790130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:33.790631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:33.791233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] PATH_INFO: `/v3/users/e678b8201f534d999ebfd7c2ecb836f7` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:33.792587 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e678b8201f534d999ebfd7c2ecb836f7)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:33.795364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:33.882586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e678b8201f534d999ebfd7c2ecb836f7'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:33.883327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e678b8201f534d999ebfd7c2ecb836f7'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:33.895433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e678b8201f534d999ebfd7c2ecb836f7'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:33.913078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e678b8201f534d999ebfd7c2ecb836f7'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:33.924567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e678b8201f534d999ebfd7c2ecb836f7'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:33.931779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1076/2148] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:54:33 2022] DELETE /identity/v3/users/e678b8201f534d999ebfd7c2ecb836f7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:33.936775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca853c3d-cdd1-406b-9f77-81fb93defebe None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:33.937069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca853c3d-cdd1-406b-9f77-81fb93defebe None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:33.937347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca853c3d-cdd1-406b-9f77-81fb93defebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:33.958249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ca853c3d-cdd1-406b-9f77-81fb93defebe None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:33.959522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ca853c3d-cdd1-406b-9f77-81fb93defebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:34.046616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1073/2149] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:34.056878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:34.058863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:34.099922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:34.141198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66457cfb-e5b4-4f3c-a2be-fc800cb75991 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:34.145128 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eec4db-3bf8-4b19-82e0-9d2100c480a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:34.145857 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eec4db-3bf8-4b19-82e0-9d2100c480a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:34.146565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eec4db-3bf8-4b19-82e0-9d2100c480a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:34.146565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eec4db-3bf8-4b19-82e0-9d2100c480a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:34.157388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eec4db-3bf8-4b19-82e0-9d2100c480a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:34.159009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eec4db-3bf8-4b19-82e0-9d2100c480a2 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:34.174722 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1077/2150] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:54:34.676184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca853c3d-cdd1-406b-9f77-81fb93defebe None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:34.678451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ca853c3d-cdd1-406b-9f77-81fb93defebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:34.693442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca853c3d-cdd1-406b-9f77-81fb93defebe None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:34.700214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:34.701789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:34.702375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:34.702744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] PATH_INFO: `/v3/projects/a3cf9f0fb1234fbcb0156092edd34bb5` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:34.703971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3cf9f0fb1234fbcb0156092edd34bb5)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:34.706656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:34.785369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3cf9f0fb1234fbcb0156092edd34bb5', 'request_id': 'req-93f1be45-c9b4-4127-a234-a1784f5ef587'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:34.796014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3cf9f0fb1234fbcb0156092edd34bb5', 'request_id': 'req-93f1be45-c9b4-4127-a234-a1784f5ef587'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:34.805038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3cf9f0fb1234fbcb0156092edd34bb5', 'request_id': 'req-93f1be45-c9b4-4127-a234-a1784f5ef587'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:34.819392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] The token cache is being invalidate because project a3cf9f0fb1234fbcb0156092edd34bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:54:34.819779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:34.820723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-93f1be45-c9b4-4127-a234-a1784f5ef587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:34.822670 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1074/2151] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:54:34 2022] DELETE /identity/v3/projects/a3cf9f0fb1234fbcb0156092edd34bb5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:54:34.828420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801157f-d633-4bed-9fdc-ab018a992784 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:34.828869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801157f-d633-4bed-9fdc-ab018a992784 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:34.829058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801157f-d633-4bed-9fdc-ab018a992784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:34.852476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a801157f-d633-4bed-9fdc-ab018a992784 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:34.853713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a801157f-d633-4bed-9fdc-ab018a992784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:34.932146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1078/2152] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:54:34.940221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab141e2-5c23-45aa-8a1b-74929c38fecc None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:34.940511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab141e2-5c23-45aa-8a1b-74929c38fecc None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:34.940805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab141e2-5c23-45aa-8a1b-74929c38fecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:34.962521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-0ab141e2-5c23-45aa-8a1b-74929c38fecc None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:34.963727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-0ab141e2-5c23-45aa-8a1b-74929c38fecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:34.974612 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1075/2153] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:54:34.978719 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801157f-d633-4bed-9fdc-ab018a992784 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:34.993292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801157f-d633-4bed-9fdc-ab018a992784 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:35.003126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba36783-5045-4b01-b3eb-031d854149bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:35.003932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba36783-5045-4b01-b3eb-031d854149bd admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.004345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba36783-5045-4b01-b3eb-031d854149bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.005255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba36783-5045-4b01-b3eb-031d854149bd admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.005835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba36783-5045-4b01-b3eb-031d854149bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:54:35.006293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba36783-5045-4b01-b3eb-031d854149bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:35.008905 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba36783-5045-4b01-b3eb-031d854149bd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:35.031237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1079/2154] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:54:35.035692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab141e2-5c23-45aa-8a1b-74929c38fecc None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:35.045352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab141e2-5c23-45aa-8a1b-74929c38fecc None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:35.050393 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db705da2-d78c-4b8f-8dd5-070e05605c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:35.051070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db705da2-d78c-4b8f-8dd5-070e05605c12 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.051568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db705da2-d78c-4b8f-8dd5-070e05605c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.051743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db705da2-d78c-4b8f-8dd5-070e05605c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.052559 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db705da2-d78c-4b8f-8dd5-070e05605c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:35.054858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db705da2-d78c-4b8f-8dd5-070e05605c12 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:35.085793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:35.099914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1076/2155] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:54:35 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:35.101110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba36783-5045-4b01-b3eb-031d854149bd admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:35.115996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba36783-5045-4b01-b3eb-031d854149bd admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:35.183013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049ecda-a1b5-48e3-a61d-acf2a7bb4744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:35.183779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ecda-a1b5-48e3-a61d-acf2a7bb4744 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.184213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ecda-a1b5-48e3-a61d-acf2a7bb4744 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.184614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ecda-a1b5-48e3-a61d-acf2a7bb4744 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.185989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049ecda-a1b5-48e3-a61d-acf2a7bb4744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:35.188109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049ecda-a1b5-48e3-a61d-acf2a7bb4744 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:35.223071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1080/2156] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:54:35 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:35.227175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db705da2-d78c-4b8f-8dd5-070e05605c12 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:35.240563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db705da2-d78c-4b8f-8dd5-070e05605c12 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:35.245784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d321e3-628b-4888-9959-70220d0a4d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:35.246576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d321e3-628b-4888-9959-70220d0a4d13 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.246914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d321e3-628b-4888-9959-70220d0a4d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.247331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d321e3-628b-4888-9959-70220d0a4d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.248668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d321e3-628b-4888-9959-70220d0a4d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:35.250480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d321e3-628b-4888-9959-70220d0a4d13 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:35.261579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089b482-946b-45cc-81fa-c54a8485e3f8 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.261880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089b482-946b-45cc-81fa-c54a8485e3f8 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.262170 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089b482-946b-45cc-81fa-c54a8485e3f8 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.266069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1077/2157] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:35.266282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1081/2158] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:54:35.267026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d321e3-628b-4888-9959-70220d0a4d13 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:35.281598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d321e3-628b-4888-9959-70220d0a4d13 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:35.283015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb43a6-98ca-4964-b62f-0374d82a5df3 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.283487 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb43a6-98ca-4964-b62f-0374d82a5df3 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.283974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb43a6-98ca-4964-b62f-0374d82a5df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.287290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fa36e-148a-477b-aede-c5d42ca12741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:35.288048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fa36e-148a-477b-aede-c5d42ca12741 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.288327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fa36e-148a-477b-aede-c5d42ca12741 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.288689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fa36e-148a-477b-aede-c5d42ca12741 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.293791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fa36e-148a-477b-aede-c5d42ca12741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:35.293791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fa36e-148a-477b-aede-c5d42ca12741 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:35.309418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1078/2159] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:35.311789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fa36e-148a-477b-aede-c5d42ca12741 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:35.318824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-93eb43a6-98ca-4964-b62f-0374d82a5df3 None None] MFA Rules not processed for user `ebc01c87551044818e1ccbb46b8e7340`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:35.322856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-93eb43a6-98ca-4964-b62f-0374d82a5df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:35.335603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fa36e-148a-477b-aede-c5d42ca12741 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:35.349578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a68a5a-62a4-4ad6-a68c-77e849480cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:35.349973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a68a5a-62a4-4ad6-a68c-77e849480cba admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.350280 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a68a5a-62a4-4ad6-a68c-77e849480cba admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.366757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a68a5a-62a4-4ad6-a68c-77e849480cba admin admin] PATH_INFO: `/v3/projects/55cd3747145043a59a6c3813b8823992/users/80e9f529289c46ed927595f03abbf3a5/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.402895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a68a5a-62a4-4ad6-a68c-77e849480cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cd3747145043a59a6c3813b8823992, user_id=80e9f529289c46ed927595f03abbf3a5, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:35.407293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a68a5a-62a4-4ad6-a68c-77e849480cba admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:35.419857 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:35.430292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1079/2160] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:54:35 2022] PUT /identity/v3/projects/55cd3747145043a59a6c3813b8823992/users/80e9f529289c46ed927595f03abbf3a5/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:35.431705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a68a5a-62a4-4ad6-a68c-77e849480cba admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:35.433399 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-72a68a5a-62a4-4ad6-a68c-77e849480cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:35.447748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1082/2161] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:54:35.449195 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c09a6-08a2-44d8-8a6d-9d3b90268497 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.449595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c09a6-08a2-44d8-8a6d-9d3b90268497 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.449942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c09a6-08a2-44d8-8a6d-9d3b90268497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.483506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-009c09a6-08a2-44d8-8a6d-9d3b90268497 None None] MFA Rules not processed for user `80e9f529289c46ed927595f03abbf3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:35.485260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-009c09a6-08a2-44d8-8a6d-9d3b90268497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:35.516523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a68a5a-62a4-4ad6-a68c-77e849480cba admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:35.522773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:35.523414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.523689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.523954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] PATH_INFO: `/v3/users/0bdbf201476a437eac34a809e137c379` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.525273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bdbf201476a437eac34a809e137c379)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:35.527666 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:35.601058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1083/2162] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:35.602394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d242a70-1cbe-4d2e-aa2f-ead7aed46704 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.602723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d242a70-1cbe-4d2e-aa2f-ead7aed46704 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.602987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d242a70-1cbe-4d2e-aa2f-ead7aed46704 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.604285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1084/2163] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 3 14:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:54:35.611294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bdbf201476a437eac34a809e137c379'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:35.611731 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7c580-cd4c-43f0-bb2a-fea4ed00bebd None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.612042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7c580-cd4c-43f0-bb2a-fea4ed00bebd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.612195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bdbf201476a437eac34a809e137c379'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:35.612491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7c580-cd4c-43f0-bb2a-fea4ed00bebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.620946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bdbf201476a437eac34a809e137c379'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:35.632595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bdbf201476a437eac34a809e137c379'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:35.633342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bdbf201476a437eac34a809e137c379'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:35.650768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ecc7c580-cd4c-43f0-bb2a-fea4ed00bebd None None] MFA Rules not processed for user `ebc01c87551044818e1ccbb46b8e7340`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:35.652206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ecc7c580-cd4c-43f0-bb2a-fea4ed00bebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:35.655419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1080/2164] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:54:35 2022] DELETE /identity/v3/users/0bdbf201476a437eac34a809e137c379 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:35.656063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:35.657428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:35.710405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:35.755631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1085/2165] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 3 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:35.757436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7c580-cd4c-43f0-bb2a-fea4ed00bebd None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:35.767203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9647c4-c25f-4a99-ab4b-4f9492556132 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:35.774197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e61d2e-bab0-4a17-8bc8-3290a45aaefa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:35.775571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e61d2e-bab0-4a17-8bc8-3290a45aaefa service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.776140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e61d2e-bab0-4a17-8bc8-3290a45aaefa service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.778033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e61d2e-bab0-4a17-8bc8-3290a45aaefa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.778225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7c580-cd4c-43f0-bb2a-fea4ed00bebd None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:35.781956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d2e61d2e-bab0-4a17-8bc8-3290a45aaefa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:35.788202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd3b06-7541-4b94-a918-6e72af67cc39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:35.788982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3b06-7541-4b94-a918-6e72af67cc39 service placement] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:35.789340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3b06-7541-4b94-a918-6e72af67cc39 service placement] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:35.789653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3b06-7541-4b94-a918-6e72af67cc39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:35.801818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6fdd3b06-7541-4b94-a918-6e72af67cc39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:35.827355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd3b06-7541-4b94-a918-6e72af67cc39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:35.830404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd3b06-7541-4b94-a918-6e72af67cc39 service placement] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:35.857229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1086/2166] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:54:35.898253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e61d2e-bab0-4a17-8bc8-3290a45aaefa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:35.899788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e61d2e-bab0-4a17-8bc8-3290a45aaefa service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:35.910652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1081/2167] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:54:35.941797 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd3b06-7541-4b94-a918-6e72af67cc39 service placement] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:35.943320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6fdd3b06-7541-4b94-a918-6e72af67cc39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:35.949328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e61d2e-bab0-4a17-8bc8-3290a45aaefa service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:35.951474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d2e61d2e-bab0-4a17-8bc8-3290a45aaefa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:36.045489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:36.052184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e61d2e-bab0-4a17-8bc8-3290a45aaefa service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:36.058593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:36.059232 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:36.059517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:36.059791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] PATH_INFO: `/v3/projects/f1d546ddcaca4842a7c598b33ff06ff5` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:36.061876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d546ddcaca4842a7c598b33ff06ff5)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:36.065217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:36.066936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd3b06-7541-4b94-a918-6e72af67cc39 service placement] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:36.073436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:36.073968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:36.074320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:36.075037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] PATH_INFO: `/v3/users/80e9f529289c46ed927595f03abbf3a5` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:36.077450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80e9f529289c46ed927595f03abbf3a5)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:36.079451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:36.130802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d546ddcaca4842a7c598b33ff06ff5', 'request_id': 'req-7e78173b-163b-4bb6-86f0-8b07a88b6e30'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.144863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d546ddcaca4842a7c598b33ff06ff5', 'request_id': 'req-7e78173b-163b-4bb6-86f0-8b07a88b6e30'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.151275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80e9f529289c46ed927595f03abbf3a5'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.152008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80e9f529289c46ed927595f03abbf3a5'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.155436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d546ddcaca4842a7c598b33ff06ff5', 'request_id': 'req-7e78173b-163b-4bb6-86f0-8b07a88b6e30'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.166395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80e9f529289c46ed927595f03abbf3a5'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.176611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] The token cache is being invalidate because project f1d546ddcaca4842a7c598b33ff06ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:54:36.176900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.177744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.179985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1082/2168] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:54:35 2022] DELETE /identity/v3/projects/f1d546ddcaca4842a7c598b33ff06ff5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:54:36.184128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:36.188212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:36.202873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80e9f529289c46ed927595f03abbf3a5'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.219661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-435650d1-60c5-480a-a56e-67bc9ba74796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80e9f529289c46ed927595f03abbf3a5'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.228971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1087/2169] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:54:35 2022] DELETE /identity/v3/users/80e9f529289c46ed927595f03abbf3a5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:36.238367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072c975-f14a-4d4a-8a54-61fff94dc2ad None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:36.238708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:36.238834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072c975-f14a-4d4a-8a54-61fff94dc2ad None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:36.239520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072c975-f14a-4d4a-8a54-61fff94dc2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:36.281545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-5072c975-f14a-4d4a-8a54-61fff94dc2ad None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:36.282801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5072c975-f14a-4d4a-8a54-61fff94dc2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:36.336947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78173b-163b-4bb6-86f0-8b07a88b6e30 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:36.408966 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1088/2170] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:36.426167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072c975-f14a-4d4a-8a54-61fff94dc2ad None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:36.427697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5072c975-f14a-4d4a-8a54-61fff94dc2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:36.489619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:36.492200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:36.492200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:36.492200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] PATH_INFO: `/v3/users/e2416ea6b67b49cb915e0b3fca243832` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:36.495572 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2416ea6b67b49cb915e0b3fca243832)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:36.497768 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:36.536458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072c975-f14a-4d4a-8a54-61fff94dc2ad None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:36.542039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aced9-4ec6-4298-9e91-9c88f5ef9eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:36.542822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aced9-4ec6-4298-9e91-9c88f5ef9eee service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:36.543148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aced9-4ec6-4298-9e91-9c88f5ef9eee service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:36.543405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aced9-4ec6-4298-9e91-9c88f5ef9eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:36.557289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4aced9-4ec6-4298-9e91-9c88f5ef9eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:36.573301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4aced9-4ec6-4298-9e91-9c88f5ef9eee service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:36.597945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2416ea6b67b49cb915e0b3fca243832'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.598632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2416ea6b67b49cb915e0b3fca243832'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.608526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1089/2171] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:54:36.610561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2416ea6b67b49cb915e0b3fca243832'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.620132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2416ea6b67b49cb915e0b3fca243832'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.620913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2416ea6b67b49cb915e0b3fca243832'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:36.645577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1083/2172] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:54:36 2022] DELETE /identity/v3/users/e2416ea6b67b49cb915e0b3fca243832 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:54:37.230839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aced9-4ec6-4298-9e91-9c88f5ef9eee service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:37.240723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aced9-4ec6-4298-9e91-9c88f5ef9eee service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:37.253568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:37.318926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:37.319983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.320280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.320560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] PATH_INFO: `/v3/projects/9b28232f58b043199222b1a1b3b589a9` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.322062 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b28232f58b043199222b1a1b3b589a9)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:37.324210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:37.381026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:37.384526 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:37.384792 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b28232f58b043199222b1a1b3b589a9', 'request_id': 'req-c6b2a955-674f-4124-8d27-9befdebb0222'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:37.406863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b28232f58b043199222b1a1b3b589a9', 'request_id': 'req-c6b2a955-674f-4124-8d27-9befdebb0222'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:37.417312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b28232f58b043199222b1a1b3b589a9', 'request_id': 'req-c6b2a955-674f-4124-8d27-9befdebb0222'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:37.419334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f172f-0b23-45ef-834b-9bc9a0646a9b admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:37.430619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] The token cache is being invalidate because project 9b28232f58b043199222b1a1b3b589a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:54:37.430935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:37.431636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c6b2a955-674f-4124-8d27-9befdebb0222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:37.433605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1090/2173] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:54:37 2022] DELETE /identity/v3/projects/9b28232f58b043199222b1a1b3b589a9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:37.438814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5827a0-5da7-439a-9219-b45869d16b56 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.439104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5827a0-5da7-439a-9219-b45869d16b56 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.439381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5827a0-5da7-439a-9219-b45869d16b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.472324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-7c5827a0-5da7-439a-9219-b45869d16b56 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:37.473681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7c5827a0-5da7-439a-9219-b45869d16b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:37.481046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:37.500125 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:37.500984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.501101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.501294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] PATH_INFO: `/v3/projects/55cd3747145043a59a6c3813b8823992` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.503613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55cd3747145043a59a6c3813b8823992)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:37.505583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:37.540909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1091/2174] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:37.551101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60fb9d-1329-47fc-96f1-b854864cc86b None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.551389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60fb9d-1329-47fc-96f1-b854864cc86b None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.551666 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60fb9d-1329-47fc-96f1-b854864cc86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.568339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55cd3747145043a59a6c3813b8823992', 'request_id': 'req-15d707e3-4122-4566-8d02-f0f09eb8c8b3'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:37.574591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-4c60fb9d-1329-47fc-96f1-b854864cc86b None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:37.575971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4c60fb9d-1329-47fc-96f1-b854864cc86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:37.578893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55cd3747145043a59a6c3813b8823992', 'request_id': 'req-15d707e3-4122-4566-8d02-f0f09eb8c8b3'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:37.585996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55cd3747145043a59a6c3813b8823992', 'request_id': 'req-15d707e3-4122-4566-8d02-f0f09eb8c8b3'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:37.605233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] The token cache is being invalidate because project 55cd3747145043a59a6c3813b8823992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:54:37.605566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:37.606257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15d707e3-4122-4566-8d02-f0f09eb8c8b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:37.608931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1084/2175] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:54:37 2022] DELETE /identity/v3/projects/55cd3747145043a59a6c3813b8823992 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:37.614363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f29c3f-60b8-436e-a37b-e9918c91ae66 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.614654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f29c3f-60b8-436e-a37b-e9918c91ae66 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.615206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f29c3f-60b8-436e-a37b-e9918c91ae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.639404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d7f29c3f-60b8-436e-a37b-e9918c91ae66 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:37.641248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:37.642255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d7f29c3f-60b8-436e-a37b-e9918c91ae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:37.672354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1092/2176] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:37.673331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60fb9d-1329-47fc-96f1-b854864cc86b None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:37.687086 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60fb9d-1329-47fc-96f1-b854864cc86b None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:37.692774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e6036-d5b3-4704-b361-f271e44a8c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:37.694131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e6036-d5b3-4704-b361-f271e44a8c6e admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.694996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e6036-d5b3-4704-b361-f271e44a8c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.695309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e6036-d5b3-4704-b361-f271e44a8c6e admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.696003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e6036-d5b3-4704-b361-f271e44a8c6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:54:37.696317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e6036-d5b3-4704-b361-f271e44a8c6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:37.698066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e6036-d5b3-4704-b361-f271e44a8c6e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:37.723415 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1093/2177] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:54:37.727869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e6036-d5b3-4704-b361-f271e44a8c6e admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:37.745081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e6036-d5b3-4704-b361-f271e44a8c6e admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:37.745648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1085/2178] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:37.754092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9cec-2181-49cd-b5ef-808316bc2255 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.754400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9cec-2181-49cd-b5ef-808316bc2255 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.754711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9cec-2181-49cd-b5ef-808316bc2255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.762659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13250c-0235-4eaa-890d-40386dc5a248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:37.764458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13250c-0235-4eaa-890d-40386dc5a248 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.764742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13250c-0235-4eaa-890d-40386dc5a248 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.765015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13250c-0235-4eaa-890d-40386dc5a248 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.777276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13250c-0235-4eaa-890d-40386dc5a248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:37.781528 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13250c-0235-4eaa-890d-40386dc5a248 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:37.785036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c08a9cec-2181-49cd-b5ef-808316bc2255 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:37.786398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c08a9cec-2181-49cd-b5ef-808316bc2255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:37.797029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1086/2179] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:37.802322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a9cec-2181-49cd-b5ef-808316bc2255 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:37.816050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a9cec-2181-49cd-b5ef-808316bc2255 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:37.821100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572365e-38ba-41fa-941c-ed74b20ef630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:37.822040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572365e-38ba-41fa-941c-ed74b20ef630 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.822328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572365e-38ba-41fa-941c-ed74b20ef630 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.822623 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572365e-38ba-41fa-941c-ed74b20ef630 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.823270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572365e-38ba-41fa-941c-ed74b20ef630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:54:37.823800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572365e-38ba-41fa-941c-ed74b20ef630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:37.826148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572365e-38ba-41fa-941c-ed74b20ef630 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:37.828367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1094/2180] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:54:37 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:37.832040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13250c-0235-4eaa-890d-40386dc5a248 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:37.845892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1087/2181] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:54:37.850314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572365e-38ba-41fa-941c-ed74b20ef630 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:37.863422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572365e-38ba-41fa-941c-ed74b20ef630 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:37.960520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13250c-0235-4eaa-890d-40386dc5a248 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:37.974511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fd0b8-74fc-4e23-bcb7-21d8f1b6eca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:37.975490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fd0b8-74fc-4e23-bcb7-21d8f1b6eca7 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.975934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fd0b8-74fc-4e23-bcb7-21d8f1b6eca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.976393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fd0b8-74fc-4e23-bcb7-21d8f1b6eca7 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.977217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fd0b8-74fc-4e23-bcb7-21d8f1b6eca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:37.977679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:37.980602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fd0b8-74fc-4e23-bcb7-21d8f1b6eca7 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:37.989433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dc41a-95fa-46c2-ab90-457c9c3b64f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:37.990858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dc41a-95fa-46c2-ab90-457c9c3b64f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:37.991214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dc41a-95fa-46c2-ab90-457c9c3b64f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:37.991496 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dc41a-95fa-46c2-ab90-457c9c3b64f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:37.992364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dc41a-95fa-46c2-ab90-457c9c3b64f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:37.994750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dc41a-95fa-46c2-ab90-457c9c3b64f5 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:38.011576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1095/2182] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:54:37 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:38.015785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fd0b8-74fc-4e23-bcb7-21d8f1b6eca7 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:38.028658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fd0b8-74fc-4e23-bcb7-21d8f1b6eca7 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:38.031318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1088/2183] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:54:37 2022] POST /identity/v3/projects => generated 391 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:38.042562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dc41a-95fa-46c2-ab90-457c9c3b64f5 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:38.127714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208b02-d251-4b02-9c71-67dedb39f1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:38.128681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208b02-d251-4b02-9c71-67dedb39f1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.129228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208b02-d251-4b02-9c71-67dedb39f1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.129529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208b02-d251-4b02-9c71-67dedb39f1b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.130211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1208b02-d251-4b02-9c71-67dedb39f1b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:38.132063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1208b02-d251-4b02-9c71-67dedb39f1b2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:38.148541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1096/2184] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:54:38.150256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dc41a-95fa-46c2-ab90-457c9c3b64f5 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:38.152645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208b02-d251-4b02-9c71-67dedb39f1b2 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:38.155771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d727d-2fdf-4508-903b-42e7afa980d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:38.156600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d727d-2fdf-4508-903b-42e7afa980d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.156937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d727d-2fdf-4508-903b-42e7afa980d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.157340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d727d-2fdf-4508-903b-42e7afa980d0 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.158085 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d727d-2fdf-4508-903b-42e7afa980d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:38.160085 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d727d-2fdf-4508-903b-42e7afa980d0 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:38.162901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208b02-d251-4b02-9c71-67dedb39f1b2 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:38.167550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b859d-21e3-4e88-829d-5c71fa5d364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:38.168151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b859d-21e3-4e88-829d-5c71fa5d364d admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.168433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b859d-21e3-4e88-829d-5c71fa5d364d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.168703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b859d-21e3-4e88-829d-5c71fa5d364d admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.169391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b859d-21e3-4e88-829d-5c71fa5d364d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:38.171061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b859d-21e3-4e88-829d-5c71fa5d364d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:38.182260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1097/2185] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:54:38.183774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1089/2186] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:54:38 2022] POST /identity/v3/users => generated 296 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:38.186704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b859d-21e3-4e88-829d-5c71fa5d364d admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:38.188290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d727d-2fdf-4508-903b-42e7afa980d0 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:38.196593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b859d-21e3-4e88-829d-5c71fa5d364d admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:38.199692 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d727d-2fdf-4508-903b-42e7afa980d0 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:38.202584 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c69deb-a8e6-404f-817b-c58fbeb855a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:38.203266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c69deb-a8e6-404f-817b-c58fbeb855a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.203546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c69deb-a8e6-404f-817b-c58fbeb855a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.203820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c69deb-a8e6-404f-817b-c58fbeb855a9 admin admin] PATH_INFO: `/v3/projects/147d53f570a14be9abe22e3b7f828806/users/e4eff18b62be45978e2c3eedefff9fb6/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.206540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347ac67-2623-4d9c-93cb-9a8af795df43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:38.208131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347ac67-2623-4d9c-93cb-9a8af795df43 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.208414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347ac67-2623-4d9c-93cb-9a8af795df43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.208691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347ac67-2623-4d9c-93cb-9a8af795df43 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.209342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347ac67-2623-4d9c-93cb-9a8af795df43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:38.210975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347ac67-2623-4d9c-93cb-9a8af795df43 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:38.219220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c69deb-a8e6-404f-817b-c58fbeb855a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147d53f570a14be9abe22e3b7f828806, user_id=e4eff18b62be45978e2c3eedefff9fb6, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:38.222659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1090/2187] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:54:38.222789 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c69deb-a8e6-404f-817b-c58fbeb855a9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:38.226898 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347ac67-2623-4d9c-93cb-9a8af795df43 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:38.237046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1098/2188] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:54:38 2022] PUT /identity/v3/projects/147d53f570a14be9abe22e3b7f828806/users/e4eff18b62be45978e2c3eedefff9fb6/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:54:38.246442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347ac67-2623-4d9c-93cb-9a8af795df43 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:38.248020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e99975-c125-439a-8f1a-5e7b553ec1d2 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.248314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e99975-c125-439a-8f1a-5e7b553ec1d2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.248588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e99975-c125-439a-8f1a-5e7b553ec1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.257947 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:38.287733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-02e99975-c125-439a-8f1a-5e7b553ec1d2 None None] MFA Rules not processed for user `e4eff18b62be45978e2c3eedefff9fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:38.288982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-02e99975-c125-439a-8f1a-5e7b553ec1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:38.315649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac61e9e-d584-4c08-8609-1208e3986d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:38.316272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac61e9e-d584-4c08-8609-1208e3986d5e admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.316554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac61e9e-d584-4c08-8609-1208e3986d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.316825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac61e9e-d584-4c08-8609-1208e3986d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.317485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac61e9e-d584-4c08-8609-1208e3986d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:38.319174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac61e9e-d584-4c08-8609-1208e3986d5e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:38.331795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1091/2189] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:54:38.335793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac61e9e-d584-4c08-8609-1208e3986d5e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:38.347468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac61e9e-d584-4c08-8609-1208e3986d5e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:38.352374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c7f2f-409d-4fdd-afd1-8a91cdd17c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:38.353203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c7f2f-409d-4fdd-afd1-8a91cdd17c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.353491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c7f2f-409d-4fdd-afd1-8a91cdd17c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.353951 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c7f2f-409d-4fdd-afd1-8a91cdd17c9d admin admin] PATH_INFO: `/v3/projects/880df2606cef440787efc6290290f037/users/7e6a94563333495881ef5ee28629f4e0/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.369262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1099/2190] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:38.371758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c7f2f-409d-4fdd-afd1-8a91cdd17c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=880df2606cef440787efc6290290f037, user_id=7e6a94563333495881ef5ee28629f4e0, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:38.375409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c7f2f-409d-4fdd-afd1-8a91cdd17c9d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:38.378679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e99975-c125-439a-8f1a-5e7b553ec1d2 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:38.380715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-02e99975-c125-439a-8f1a-5e7b553ec1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:38.391784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1092/2191] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:54:38 2022] PUT /identity/v3/projects/880df2606cef440787efc6290290f037/users/7e6a94563333495881ef5ee28629f4e0/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:54:38.402806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010b0bf-67de-4363-b845-6426cb7e9250 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.403101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010b0bf-67de-4363-b845-6426cb7e9250 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.403391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010b0bf-67de-4363-b845-6426cb7e9250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.445805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-9010b0bf-67de-4363-b845-6426cb7e9250 None None] MFA Rules not processed for user `7e6a94563333495881ef5ee28629f4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:38.447734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9010b0bf-67de-4363-b845-6426cb7e9250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:38.462342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e99975-c125-439a-8f1a-5e7b553ec1d2 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:38.470943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:38.524645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec1871-d9ac-4fee-84f9-cc19e8812344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:38.529555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1093/2192] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:54:38.529734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec1871-d9ac-4fee-84f9-cc19e8812344 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.529734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec1871-d9ac-4fee-84f9-cc19e8812344 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.529734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec1871-d9ac-4fee-84f9-cc19e8812344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.538398 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010b0bf-67de-4363-b845-6426cb7e9250 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:38.539856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9010b0bf-67de-4363-b845-6426cb7e9250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:38.552688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010b0bf-67de-4363-b845-6426cb7e9250 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:38.556745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40a556-d670-4301-ab4e-958874c01731 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:38.557354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40a556-d670-4301-ab4e-958874c01731 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:38.557745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40a556-d670-4301-ab4e-958874c01731 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:38.558290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40a556-d670-4301-ab4e-958874c01731 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:38.567199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40a556-d670-4301-ab4e-958874c01731 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:38.568656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40a556-d670-4301-ab4e-958874c01731 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:38.593575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1094/2193] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:54:38.602627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec1871-d9ac-4fee-84f9-cc19e8812344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:38.604253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec1871-d9ac-4fee-84f9-cc19e8812344 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:38.616150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1100/2194] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:54:48.574103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=24 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:48.611022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:56.443701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40a556-d670-4301-ab4e-958874c01731 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:56.504577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40a556-d670-4301-ab4e-958874c01731 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:56.509034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:56.509678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:56.509961 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:56.510234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] PATH_INFO: `/v3/users/e4eff18b62be45978e2c3eedefff9fb6` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:56.511345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4eff18b62be45978e2c3eedefff9fb6)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:56.513272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:56.557020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4eff18b62be45978e2c3eedefff9fb6'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:56.557620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4eff18b62be45978e2c3eedefff9fb6'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:56.565090 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4eff18b62be45978e2c3eedefff9fb6'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:56.572203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4eff18b62be45978e2c3eedefff9fb6'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:56.572824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4eff18b62be45978e2c3eedefff9fb6'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:56.590438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1095/2195] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:54:56 2022] DELETE /identity/v3/users/e4eff18b62be45978e2c3eedefff9fb6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:56.594957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97313-4c0e-4069-99ef-b69819017eb6 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:56.595413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97313-4c0e-4069-99ef-b69819017eb6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:56.595535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97313-4c0e-4069-99ef-b69819017eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:56.614832 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-21a97313-4c0e-4069-99ef-b69819017eb6 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:56.616026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-21a97313-4c0e-4069-99ef-b69819017eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:56.684095 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1101/2196] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:56.695343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:56.696780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:56.763477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b895c1-c246-488d-9aee-224de6920223 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:56.767319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4fb7d-e3b2-4208-8cd6-946c6b2a7a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:56.767929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4fb7d-e3b2-4208-8cd6-946c6b2a7a81 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:56.768208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4fb7d-e3b2-4208-8cd6-946c6b2a7a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:56.768479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4fb7d-e3b2-4208-8cd6-946c6b2a7a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:56.779420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4fb7d-e3b2-4208-8cd6-946c6b2a7a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:56.780868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4fb7d-e3b2-4208-8cd6-946c6b2a7a81 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:56.790555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1096/2197] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:54:57.141592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97313-4c0e-4069-99ef-b69819017eb6 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:57.143097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-21a97313-4c0e-4069-99ef-b69819017eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:57.155702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97313-4c0e-4069-99ef-b69819017eb6 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:57.160568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:57.161204 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.161486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.161792 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] PATH_INFO: `/v3/projects/147d53f570a14be9abe22e3b7f828806` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.162905 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=147d53f570a14be9abe22e3b7f828806)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:57.164815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:57.212055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147d53f570a14be9abe22e3b7f828806', 'request_id': 'req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:57.219924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147d53f570a14be9abe22e3b7f828806', 'request_id': 'req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:57.226019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147d53f570a14be9abe22e3b7f828806', 'request_id': 'req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:57.236126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] The token cache is being invalidate because project 147d53f570a14be9abe22e3b7f828806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:54:57.236416 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:57.237057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-517c9043-9cb0-4d38-8adf-5df3d12d9ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:54:57.238545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1102/2198] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:54:57 2022] DELETE /identity/v3/projects/147d53f570a14be9abe22e3b7f828806 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:57.243416 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b54ca1-e327-4e42-97b6-f132c73da0ff None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.243697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b54ca1-e327-4e42-97b6-f132c73da0ff None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.243968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b54ca1-e327-4e42-97b6-f132c73da0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.263434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b0b54ca1-e327-4e42-97b6-f132c73da0ff None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:57.264654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b0b54ca1-e327-4e42-97b6-f132c73da0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:57.281605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:57.325185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1097/2199] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:57.333369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1e5f5-af1d-43a9-9565-38ca42d59798 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.333691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1e5f5-af1d-43a9-9565-38ca42d59798 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.333969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1e5f5-af1d-43a9-9565-38ca42d59798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.354859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-5ae1e5f5-af1d-43a9-9565-38ca42d59798 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:57.356050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5ae1e5f5-af1d-43a9-9565-38ca42d59798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:57.363743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1103/2200] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:57.367720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b54ca1-e327-4e42-97b6-f132c73da0ff None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:57.377337 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b54ca1-e327-4e42-97b6-f132c73da0ff None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:57.381846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f65f57-a2bd-404d-a74f-0a37b46500ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:57.382430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f65f57-a2bd-404d-a74f-0a37b46500ba admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.382703 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f65f57-a2bd-404d-a74f-0a37b46500ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.382966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f65f57-a2bd-404d-a74f-0a37b46500ba admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.383605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f65f57-a2bd-404d-a74f-0a37b46500ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:54:57.383908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f65f57-a2bd-404d-a74f-0a37b46500ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:57.385576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f65f57-a2bd-404d-a74f-0a37b46500ba admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:57.397272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1098/2201] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:54:57.401208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1e5f5-af1d-43a9-9565-38ca42d59798 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:57.410794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1e5f5-af1d-43a9-9565-38ca42d59798 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:57.415342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6ff2a-3f2e-49cf-912f-271365b5b86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:57.415930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6ff2a-3f2e-49cf-912f-271365b5b86a admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.416207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6ff2a-3f2e-49cf-912f-271365b5b86a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.416477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6ff2a-3f2e-49cf-912f-271365b5b86a admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.417097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6ff2a-3f2e-49cf-912f-271365b5b86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:57.419031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6ff2a-3f2e-49cf-912f-271365b5b86a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:57.445344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1104/2202] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:54:57 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:57.451340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f65f57-a2bd-404d-a74f-0a37b46500ba admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:57.461024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f65f57-a2bd-404d-a74f-0a37b46500ba admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:57.516239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e1fdf-20a7-4bea-a8c9-779595440474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:57.516834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e1fdf-20a7-4bea-a8c9-779595440474 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.517109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e1fdf-20a7-4bea-a8c9-779595440474 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.517377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e1fdf-20a7-4bea-a8c9-779595440474 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.518025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e1fdf-20a7-4bea-a8c9-779595440474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:57.519940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e1fdf-20a7-4bea-a8c9-779595440474 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:57.541412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1099/2203] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:54:57 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:57.545675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6ff2a-3f2e-49cf-912f-271365b5b86a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:57.555695 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6ff2a-3f2e-49cf-912f-271365b5b86a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:57.560840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9715e3-8b3d-435c-9ec3-449eb24016ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:57.561433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9715e3-8b3d-435c-9ec3-449eb24016ea admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.561740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9715e3-8b3d-435c-9ec3-449eb24016ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.562011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9715e3-8b3d-435c-9ec3-449eb24016ea admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.562658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9715e3-8b3d-435c-9ec3-449eb24016ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:57.564238 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9715e3-8b3d-435c-9ec3-449eb24016ea admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:57.573841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1105/2204] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:54:57.578263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e1fdf-20a7-4bea-a8c9-779595440474 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:57.587863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e1fdf-20a7-4bea-a8c9-779595440474 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:57.593001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47464f-8854-410d-8d44-cdf7c8098fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:57.593613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47464f-8854-410d-8d44-cdf7c8098fde admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.593893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47464f-8854-410d-8d44-cdf7c8098fde admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.594163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47464f-8854-410d-8d44-cdf7c8098fde admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.594790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47464f-8854-410d-8d44-cdf7c8098fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:57.596354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47464f-8854-410d-8d44-cdf7c8098fde admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:57.606946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1100/2205] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:57.610977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9715e3-8b3d-435c-9ec3-449eb24016ea admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:57.620655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9715e3-8b3d-435c-9ec3-449eb24016ea admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:57.625771 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d45039-22dd-4d64-8d0c-9c7c3ae7cbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:57.626406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45039-22dd-4d64-8d0c-9c7c3ae7cbac admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.626685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45039-22dd-4d64-8d0c-9c7c3ae7cbac admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.626975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45039-22dd-4d64-8d0c-9c7c3ae7cbac admin admin] PATH_INFO: `/v3/projects/d474757100a24e62988f5c9e71c20932/users/a1aa124ad17c49b99bec18911acbd268/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.641956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d45039-22dd-4d64-8d0c-9c7c3ae7cbac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d474757100a24e62988f5c9e71c20932, user_id=a1aa124ad17c49b99bec18911acbd268, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:57.644524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d45039-22dd-4d64-8d0c-9c7c3ae7cbac admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:57.656968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1106/2206] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:54:57 2022] PUT /identity/v3/projects/d474757100a24e62988f5c9e71c20932/users/a1aa124ad17c49b99bec18911acbd268/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:54:57.666976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf2a8-f438-4776-b7e2-1b8c49cb4082 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.667256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf2a8-f438-4776-b7e2-1b8c49cb4082 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.667531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf2a8-f438-4776-b7e2-1b8c49cb4082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.697535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-38acf2a8-f438-4776-b7e2-1b8c49cb4082 None None] MFA Rules not processed for user `a1aa124ad17c49b99bec18911acbd268`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:57.702199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-38acf2a8-f438-4776-b7e2-1b8c49cb4082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:57.767361 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1101/2207] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:57.771811 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d45039-22dd-4d64-8d0c-9c7c3ae7cbac admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:57.791924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:57.834477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d45039-22dd-4d64-8d0c-9c7c3ae7cbac admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:57.839150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcf4b6-352f-4fe8-bb23-1136d8f40312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:57.840000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcf4b6-352f-4fe8-bb23-1136d8f40312 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.840281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcf4b6-352f-4fe8-bb23-1136d8f40312 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.840552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcf4b6-352f-4fe8-bb23-1136d8f40312 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.841183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcf4b6-352f-4fe8-bb23-1136d8f40312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:57.843119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcf4b6-352f-4fe8-bb23-1136d8f40312 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:57.870250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1107/2208] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:54:57 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:57.874957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf2a8-f438-4776-b7e2-1b8c49cb4082 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:57.885782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf2a8-f438-4776-b7e2-1b8c49cb4082 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:57.898586 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:57.944332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede104ca-0795-42c8-9da6-30098017fc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:57.944938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede104ca-0795-42c8-9da6-30098017fc32 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.945214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede104ca-0795-42c8-9da6-30098017fc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.945482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede104ca-0795-42c8-9da6-30098017fc32 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.946136 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede104ca-0795-42c8-9da6-30098017fc32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:57.948044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede104ca-0795-42c8-9da6-30098017fc32 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:57.969440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1102/2209] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:54:57 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:54:57.973237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcf4b6-352f-4fe8-bb23-1136d8f40312 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:57.984564 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcf4b6-352f-4fe8-bb23-1136d8f40312 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:57.990597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fe4e3-82a8-4daa-81ca-a1ed863facf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:57.991198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fe4e3-82a8-4daa-81ca-a1ed863facf2 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:57.991478 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fe4e3-82a8-4daa-81ca-a1ed863facf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:57.991744 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fe4e3-82a8-4daa-81ca-a1ed863facf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:57.992432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39fe4e3-82a8-4daa-81ca-a1ed863facf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:57.993999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39fe4e3-82a8-4daa-81ca-a1ed863facf2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:58.009142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1108/2210] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:58.011043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede104ca-0795-42c8-9da6-30098017fc32 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:58.021617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede104ca-0795-42c8-9da6-30098017fc32 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:58.026354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754f832-3766-467f-ba04-b98557a09d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:58.026965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754f832-3766-467f-ba04-b98557a09d89 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.027245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754f832-3766-467f-ba04-b98557a09d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.027514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754f832-3766-467f-ba04-b98557a09d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.028159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754f832-3766-467f-ba04-b98557a09d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:58.029813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754f832-3766-467f-ba04-b98557a09d89 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:58.040231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1103/2211] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:58.044924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fe4e3-82a8-4daa-81ca-a1ed863facf2 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:58.055088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fe4e3-82a8-4daa-81ca-a1ed863facf2 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:58.060413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8375301-1ec4-4648-9d08-d85d3a58546c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:58.061073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8375301-1ec4-4648-9d08-d85d3a58546c admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.061355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8375301-1ec4-4648-9d08-d85d3a58546c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.061667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8375301-1ec4-4648-9d08-d85d3a58546c admin admin] PATH_INFO: `/v3/projects/c0571a92c8bd4e25aa54c16083e57f45/users/bf0b8194fa644dfcafe152b12a163862/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.076675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8375301-1ec4-4648-9d08-d85d3a58546c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0571a92c8bd4e25aa54c16083e57f45, user_id=bf0b8194fa644dfcafe152b12a163862, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:58.079450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8375301-1ec4-4648-9d08-d85d3a58546c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:58.094092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1109/2212] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:54:58 2022] PUT /identity/v3/projects/c0571a92c8bd4e25aa54c16083e57f45/users/bf0b8194fa644dfcafe152b12a163862/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:54:58.104317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2aef1-f535-4ed4-8502-fa5553e1a7de None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.104604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2aef1-f535-4ed4-8502-fa5553e1a7de None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.104879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2aef1-f535-4ed4-8502-fa5553e1a7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.135526 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-14d2aef1-f535-4ed4-8502-fa5553e1a7de None None] MFA Rules not processed for user `bf0b8194fa644dfcafe152b12a163862`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:58.136736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-14d2aef1-f535-4ed4-8502-fa5553e1a7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:58.199251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1104/2213] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:58.203728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8375301-1ec4-4648-9d08-d85d3a58546c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:58.264667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8375301-1ec4-4648-9d08-d85d3a58546c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:58.269230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6293b2-8426-4b3d-bd5b-d6a144f5a158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:58.269850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6293b2-8426-4b3d-bd5b-d6a144f5a158 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.270127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6293b2-8426-4b3d-bd5b-d6a144f5a158 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.270396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6293b2-8426-4b3d-bd5b-d6a144f5a158 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.271313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6293b2-8426-4b3d-bd5b-d6a144f5a158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:58.273210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6293b2-8426-4b3d-bd5b-d6a144f5a158 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:58.301393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1110/2214] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:54:58 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:58.305555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2aef1-f535-4ed4-8502-fa5553e1a7de None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:58.316117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2aef1-f535-4ed4-8502-fa5553e1a7de None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:58.371276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3786a-c8f0-4ad2-a5ce-bf6623197120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:58.371877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3786a-c8f0-4ad2-a5ce-bf6623197120 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.372168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3786a-c8f0-4ad2-a5ce-bf6623197120 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.372451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3786a-c8f0-4ad2-a5ce-bf6623197120 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.373078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3786a-c8f0-4ad2-a5ce-bf6623197120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:58.375037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3786a-c8f0-4ad2-a5ce-bf6623197120 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:58.395151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1105/2215] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:54:58 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:54:58.399307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6293b2-8426-4b3d-bd5b-d6a144f5a158 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:58.409938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6293b2-8426-4b3d-bd5b-d6a144f5a158 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:58.414546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8723a-6945-45bd-921f-04e02f47f808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:58.415145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8723a-6945-45bd-921f-04e02f47f808 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.415424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8723a-6945-45bd-921f-04e02f47f808 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.415697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8723a-6945-45bd-921f-04e02f47f808 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.416343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8723a-6945-45bd-921f-04e02f47f808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:58.418000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8723a-6945-45bd-921f-04e02f47f808 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:58.428796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1111/2216] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:58.432814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3786a-c8f0-4ad2-a5ce-bf6623197120 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:58.442735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3786a-c8f0-4ad2-a5ce-bf6623197120 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:58.447292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a2b84-247f-43cc-a969-dfa49755719a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:58.447929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a2b84-247f-43cc-a969-dfa49755719a admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.448211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a2b84-247f-43cc-a969-dfa49755719a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.448487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a2b84-247f-43cc-a969-dfa49755719a admin admin] PATH_INFO: `/v3/projects/0e29fd6872ac462cb9f9dda86f97ec5e/users/2f8107e4fa1948b38e8c69e7b2b18815/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.460614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:58.463932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a2b84-247f-43cc-a969-dfa49755719a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e29fd6872ac462cb9f9dda86f97ec5e, user_id=2f8107e4fa1948b38e8c69e7b2b18815, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:58.467803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a2b84-247f-43cc-a969-dfa49755719a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:58.480427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1106/2217] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:54:58 2022] PUT /identity/v3/projects/0e29fd6872ac462cb9f9dda86f97ec5e/users/2f8107e4fa1948b38e8c69e7b2b18815/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:58.484860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8723a-6945-45bd-921f-04e02f47f808 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:58.494795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8723a-6945-45bd-921f-04e02f47f808 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:58.536086 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:54:58.552224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9266b80-327f-4d25-9a9d-7f7c801586c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:58.552830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9266b80-327f-4d25-9a9d-7f7c801586c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.553110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9266b80-327f-4d25-9a9d-7f7c801586c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.553380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9266b80-327f-4d25-9a9d-7f7c801586c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.554058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9266b80-327f-4d25-9a9d-7f7c801586c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:58.555633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9266b80-327f-4d25-9a9d-7f7c801586c7 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:58.565113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1112/2218] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:54:58.569345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a2b84-247f-43cc-a969-dfa49755719a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:58.580467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a2b84-247f-43cc-a969-dfa49755719a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:58.585132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d25b6-ba3b-4cf5-898e-3bccb9bb79f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:58.585818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d25b6-ba3b-4cf5-898e-3bccb9bb79f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.586101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d25b6-ba3b-4cf5-898e-3bccb9bb79f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.586375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d25b6-ba3b-4cf5-898e-3bccb9bb79f2 admin admin] PATH_INFO: `/v3/projects/0e29fd6872ac462cb9f9dda86f97ec5e/users/2f8107e4fa1948b38e8c69e7b2b18815/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.588257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d25b6-ba3b-4cf5-898e-3bccb9bb79f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e29fd6872ac462cb9f9dda86f97ec5e, user_id=2f8107e4fa1948b38e8c69e7b2b18815, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:58.591834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d25b6-ba3b-4cf5-898e-3bccb9bb79f2 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:58.601913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1107/2219] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:54:58 2022] PUT /identity/v3/projects/0e29fd6872ac462cb9f9dda86f97ec5e/users/2f8107e4fa1948b38e8c69e7b2b18815/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:54:58.612800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d096e-7ec2-48fe-9d6c-40683c321f53 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.613292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d096e-7ec2-48fe-9d6c-40683c321f53 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.613384 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d096e-7ec2-48fe-9d6c-40683c321f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.641110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-402d096e-7ec2-48fe-9d6c-40683c321f53 None None] MFA Rules not processed for user `2f8107e4fa1948b38e8c69e7b2b18815`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:54:58.642339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-402d096e-7ec2-48fe-9d6c-40683c321f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:58.708024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1113/2220] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:54:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:54:58.720239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d25b6-ba3b-4cf5-898e-3bccb9bb79f2 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:54:58.721994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-784d25b6-ba3b-4cf5-898e-3bccb9bb79f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:54:58.781133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d25b6-ba3b-4cf5-898e-3bccb9bb79f2 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:54:58.784832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a099a42-67d2-4748-87c6-2ad55421545d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:54:58.785442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a099a42-67d2-4748-87c6-2ad55421545d service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:54:58.785762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a099a42-67d2-4748-87c6-2ad55421545d service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:54:58.786034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a099a42-67d2-4748-87c6-2ad55421545d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:54:58.840745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a099a42-67d2-4748-87c6-2ad55421545d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:54:58.842823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a099a42-67d2-4748-87c6-2ad55421545d service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:54:58.851503 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1108/2221] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:54:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:55:03.226522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d096e-7ec2-48fe-9d6c-40683c321f53 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:03.227934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-402d096e-7ec2-48fe-9d6c-40683c321f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:03.296090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d096e-7ec2-48fe-9d6c-40683c321f53 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:03.301156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1d1ce-d496-4bde-baf7-53d47aee1d58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:03.301790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1d1ce-d496-4bde-baf7-53d47aee1d58 service placement] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:03.302071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1d1ce-d496-4bde-baf7-53d47aee1d58 service placement] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:03.302338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1d1ce-d496-4bde-baf7-53d47aee1d58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:03.304112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f0d1d1ce-d496-4bde-baf7-53d47aee1d58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:03.315322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1d1ce-d496-4bde-baf7-53d47aee1d58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:03.316901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1d1ce-d496-4bde-baf7-53d47aee1d58 service placement] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:03.327315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1114/2222] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:55:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:55:03.524117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a099a42-67d2-4748-87c6-2ad55421545d service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:03.525475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3a099a42-67d2-4748-87c6-2ad55421545d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:03.536420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a099a42-67d2-4748-87c6-2ad55421545d service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:03.539948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bf530-9d4c-4723-adc7-a467f5b58454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:03.540533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bf530-9d4c-4723-adc7-a467f5b58454 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:03.540807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bf530-9d4c-4723-adc7-a467f5b58454 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:03.541085 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bf530-9d4c-4723-adc7-a467f5b58454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:03.595286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bf530-9d4c-4723-adc7-a467f5b58454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:03.596710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bf530-9d4c-4723-adc7-a467f5b58454 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:03.605305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1109/2223] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:55:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:55:04.249667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1d1ce-d496-4bde-baf7-53d47aee1d58 service placement] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:04.259563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1d1ce-d496-4bde-baf7-53d47aee1d58 service placement] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:04.263296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbdd79-9ecf-43d4-a836-91d7ef821d41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:04.263896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbdd79-9ecf-43d4-a836-91d7ef821d41 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:04.264178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbdd79-9ecf-43d4-a836-91d7ef821d41 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:04.264453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbdd79-9ecf-43d4-a836-91d7ef821d41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:04.273562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbdd79-9ecf-43d4-a836-91d7ef821d41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:04.274994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbdd79-9ecf-43d4-a836-91d7ef821d41 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:04.286119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1115/2224] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:55:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:55:07.342403 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bf530-9d4c-4723-adc7-a467f5b58454 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:07.343842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e03bf530-9d4c-4723-adc7-a467f5b58454 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:07.381854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:07.417787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bf530-9d4c-4723-adc7-a467f5b58454 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:07.425436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:07.426060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:07.426530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:07.427217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] PATH_INFO: `/v3/users/7e6a94563333495881ef5ee28629f4e0` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:07.428043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e6a94563333495881ef5ee28629f4e0)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:07.430134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:07.482622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e6a94563333495881ef5ee28629f4e0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:07.483270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e6a94563333495881ef5ee28629f4e0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:07.491749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e6a94563333495881ef5ee28629f4e0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:07.499866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e6a94563333495881ef5ee28629f4e0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:07.500533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e6a94563333495881ef5ee28629f4e0'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:07.518630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1110/2225] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:55:07 2022] DELETE /identity/v3/users/7e6a94563333495881ef5ee28629f4e0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:07.523120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f967a2e-bac1-4819-a24d-24bf03c76370 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:07.523590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f967a2e-bac1-4819-a24d-24bf03c76370 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:07.523679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f967a2e-bac1-4819-a24d-24bf03c76370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:07.544499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6f967a2e-bac1-4819-a24d-24bf03c76370 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:07.545842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6f967a2e-bac1-4819-a24d-24bf03c76370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:07.599888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:07.623536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1116/2226] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:55:07.633216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:07.634655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:07.691425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dabcf4-686b-41b8-bbac-4602faa57bee admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:07.695175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fce88-ef69-4a71-a394-fee17ec59888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:07.695862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fce88-ef69-4a71-a394-fee17ec59888 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:07.696208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fce88-ef69-4a71-a394-fee17ec59888 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:07.696542 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fce88-ef69-4a71-a394-fee17ec59888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:07.706244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fce88-ef69-4a71-a394-fee17ec59888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:07.707714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fce88-ef69-4a71-a394-fee17ec59888 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:07.718228 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1111/2227] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:55:07.990950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f967a2e-bac1-4819-a24d-24bf03c76370 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:07.992398 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6f967a2e-bac1-4819-a24d-24bf03c76370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:08.008145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f967a2e-bac1-4819-a24d-24bf03c76370 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:08.013919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:08.014552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.014837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.015092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] PATH_INFO: `/v3/projects/880df2606cef440787efc6290290f037` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.016211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=880df2606cef440787efc6290290f037)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:08.018137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:08.068531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880df2606cef440787efc6290290f037', 'request_id': 'req-67ae212d-5288-403b-8b03-d533080ddd10'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:08.076938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880df2606cef440787efc6290290f037', 'request_id': 'req-67ae212d-5288-403b-8b03-d533080ddd10'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:08.083516 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880df2606cef440787efc6290290f037', 'request_id': 'req-67ae212d-5288-403b-8b03-d533080ddd10'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:08.094240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] The token cache is being invalidate because project 880df2606cef440787efc6290290f037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:55:08.094531 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:08.095172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-67ae212d-5288-403b-8b03-d533080ddd10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:08.096675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1117/2228] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:55:07 2022] DELETE /identity/v3/projects/880df2606cef440787efc6290290f037 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:08.101705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b96e7-574a-4384-af0c-f0bdfc2b9743 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.101989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b96e7-574a-4384-af0c-f0bdfc2b9743 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.102264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b96e7-574a-4384-af0c-f0bdfc2b9743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.122497 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f62b96e7-574a-4384-af0c-f0bdfc2b9743 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:08.123702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f62b96e7-574a-4384-af0c-f0bdfc2b9743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:08.173952 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:08.205334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1112/2229] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:55:08.213887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b731629-9fab-474d-8401-cdd86b654b3d None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.214899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b731629-9fab-474d-8401-cdd86b654b3d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.217805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b731629-9fab-474d-8401-cdd86b654b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.249894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b96e7-574a-4384-af0c-f0bdfc2b9743 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:08.251407 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f62b96e7-574a-4384-af0c-f0bdfc2b9743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:08.265004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-3b731629-9fab-474d-8401-cdd86b654b3d None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:08.266288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3b731629-9fab-474d-8401-cdd86b654b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:08.271932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b96e7-574a-4384-af0c-f0bdfc2b9743 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:08.279264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:08.280657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.280786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.281599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] PATH_INFO: `/v3/users/a1aa124ad17c49b99bec18911acbd268` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.285605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1aa124ad17c49b99bec18911acbd268)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:08.287560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:08.296084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1118/2230] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:55:08.300604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b731629-9fab-474d-8401-cdd86b654b3d None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:08.311071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b731629-9fab-474d-8401-cdd86b654b3d None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:08.316713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea40c30-f865-447e-a7c1-3aad684657c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:08.317698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea40c30-f865-447e-a7c1-3aad684657c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.317698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea40c30-f865-447e-a7c1-3aad684657c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.317908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea40c30-f865-447e-a7c1-3aad684657c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.318540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea40c30-f865-447e-a7c1-3aad684657c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:55:08.318847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea40c30-f865-447e-a7c1-3aad684657c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:08.320456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea40c30-f865-447e-a7c1-3aad684657c3 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:08.335067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1119/2231] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 14:55:08.341838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea40c30-f865-447e-a7c1-3aad684657c3 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:08.353948 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea40c30-f865-447e-a7c1-3aad684657c3 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:08.361915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b30c5-80c1-47f0-bfa6-627142ef94d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:08.362681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b30c5-80c1-47f0-bfa6-627142ef94d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.364984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b30c5-80c1-47f0-bfa6-627142ef94d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.365305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b30c5-80c1-47f0-bfa6-627142ef94d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.365993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b30c5-80c1-47f0-bfa6-627142ef94d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:08.369904 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b30c5-80c1-47f0-bfa6-627142ef94d0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:08.373762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1aa124ad17c49b99bec18911acbd268'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:08.374457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1aa124ad17c49b99bec18911acbd268'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:08.383177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1aa124ad17c49b99bec18911acbd268'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:08.394460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1aa124ad17c49b99bec18911acbd268'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:08.395146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f3073a75-168a-4f5d-8ecd-1172f978a841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1aa124ad17c49b99bec18911acbd268'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:08.419976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1113/2232] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:55:08 2022] DELETE /identity/v3/users/a1aa124ad17c49b99bec18911acbd268 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:08.424353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3df0b0-932e-4039-9dc0-c85843432ca8 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.425787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3df0b0-932e-4039-9dc0-c85843432ca8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.426243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3df0b0-932e-4039-9dc0-c85843432ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.435174 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1120/2233] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:55:08 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:55:08.436283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b30c5-80c1-47f0-bfa6-627142ef94d0 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:08.438156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a46b30c5-80c1-47f0-bfa6-627142ef94d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:08.474429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-6c3df0b0-932e-4039-9dc0-c85843432ca8 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:08.475963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6c3df0b0-932e-4039-9dc0-c85843432ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:08.528364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b30c5-80c1-47f0-bfa6-627142ef94d0 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:08.535722 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98c38d-de31-4bd5-8424-320f34e86e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:08.536414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98c38d-de31-4bd5-8424-320f34e86e60 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.536628 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98c38d-de31-4bd5-8424-320f34e86e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.536936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98c38d-de31-4bd5-8424-320f34e86e60 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.537574 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98c38d-de31-4bd5-8424-320f34e86e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:08.539520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98c38d-de31-4bd5-8424-320f34e86e60 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:08.566051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1121/2234] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:55:08 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:55:08.568401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1114/2235] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:55:08.572367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98c38d-de31-4bd5-8424-320f34e86e60 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:08.580581 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3df0b0-932e-4039-9dc0-c85843432ca8 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:08.582608 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98c38d-de31-4bd5-8424-320f34e86e60 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:08.583282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6c3df0b0-932e-4039-9dc0-c85843432ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:08.588461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74e995-d03b-4f07-a531-47686ff5b1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:08.589105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74e995-d03b-4f07-a531-47686ff5b1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.589385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74e995-d03b-4f07-a531-47686ff5b1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.589945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74e995-d03b-4f07-a531-47686ff5b1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.591403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74e995-d03b-4f07-a531-47686ff5b1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:08.592248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74e995-d03b-4f07-a531-47686ff5b1b5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:08.610500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1122/2236] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:55:08.611500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74e995-d03b-4f07-a531-47686ff5b1b5 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:08.616932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:08.626664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74e995-d03b-4f07-a531-47686ff5b1b5 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:08.632841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6468991-b98d-445b-8bed-c4b516e18b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:08.633954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6468991-b98d-445b-8bed-c4b516e18b6f admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.634621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6468991-b98d-445b-8bed-c4b516e18b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.635265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6468991-b98d-445b-8bed-c4b516e18b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.635932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6468991-b98d-445b-8bed-c4b516e18b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:08.637623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6468991-b98d-445b-8bed-c4b516e18b6f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:08.653534 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1123/2237] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:55:08.717104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3df0b0-932e-4039-9dc0-c85843432ca8 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:08.721683 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6468991-b98d-445b-8bed-c4b516e18b6f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:08.721835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2e31f-3fbb-46ff-b0cf-0c11fb3a7dc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:08.721835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2e31f-3fbb-46ff-b0cf-0c11fb3a7dc0 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.721835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2e31f-3fbb-46ff-b0cf-0c11fb3a7dc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.721835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2e31f-3fbb-46ff-b0cf-0c11fb3a7dc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.733850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6468991-b98d-445b-8bed-c4b516e18b6f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:08.735004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2e31f-3fbb-46ff-b0cf-0c11fb3a7dc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:08.737431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2e31f-3fbb-46ff-b0cf-0c11fb3a7dc0 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:08.738757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ecbd0-a0a0-4b40-9159-c27baefb0307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:08.739432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ecbd0-a0a0-4b40-9159-c27baefb0307 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.739709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ecbd0-a0a0-4b40-9159-c27baefb0307 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.739983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ecbd0-a0a0-4b40-9159-c27baefb0307 admin admin] PATH_INFO: `/v3/projects/8fae62571c3c402a8fcc4afaf4268310/users/7725c72e89964487843bc9bd46b8e3d2/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.749682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1115/2238] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:55:08.759442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ecbd0-a0a0-4b40-9159-c27baefb0307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fae62571c3c402a8fcc4afaf4268310, user_id=7725c72e89964487843bc9bd46b8e3d2, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:08.762603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ecbd0-a0a0-4b40-9159-c27baefb0307 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:08.780264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1124/2239] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:55:08 2022] PUT /identity/v3/projects/8fae62571c3c402a8fcc4afaf4268310/users/7725c72e89964487843bc9bd46b8e3d2/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:08.785406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b485df-c733-4486-bfaf-eef4339053e0 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.785953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b485df-c733-4486-bfaf-eef4339053e0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.786270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b485df-c733-4486-bfaf-eef4339053e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.806981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-77b485df-c733-4486-bfaf-eef4339053e0 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:08.808245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-77b485df-c733-4486-bfaf-eef4339053e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:08.813157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:08.888126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1116/2240] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:55:08.909214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ecbd0-a0a0-4b40-9159-c27baefb0307 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:08.970929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ecbd0-a0a0-4b40-9159-c27baefb0307 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:08.974546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c7656-bc20-4c74-af35-01ffa029693e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:08.975170 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c7656-bc20-4c74-af35-01ffa029693e service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:08.975447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c7656-bc20-4c74-af35-01ffa029693e service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:08.975718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c7656-bc20-4c74-af35-01ffa029693e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:08.986802 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c7656-bc20-4c74-af35-01ffa029693e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:08.989406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c7656-bc20-4c74-af35-01ffa029693e service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:08.999414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1125/2241] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 14:55:09.083761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b485df-c733-4486-bfaf-eef4339053e0 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:09.085382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-77b485df-c733-4486-bfaf-eef4339053e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:09.098503 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b485df-c733-4486-bfaf-eef4339053e0 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:09.103321 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:09.103987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:09.104304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:09.105158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] PATH_INFO: `/v3/projects/d474757100a24e62988f5c9e71c20932` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:09.105818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d474757100a24e62988f5c9e71c20932)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:09.107965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:09.164578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd474757100a24e62988f5c9e71c20932', 'request_id': 'req-967b77c9-d651-4ff4-8360-9cdb95d86f2b'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:09.172311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd474757100a24e62988f5c9e71c20932', 'request_id': 'req-967b77c9-d651-4ff4-8360-9cdb95d86f2b'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:09.179254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd474757100a24e62988f5c9e71c20932', 'request_id': 'req-967b77c9-d651-4ff4-8360-9cdb95d86f2b'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:09.190773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] The token cache is being invalidate because project d474757100a24e62988f5c9e71c20932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:55:09.191053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:09.191697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:09.193558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1117/2242] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:55:09 2022] DELETE /identity/v3/projects/d474757100a24e62988f5c9e71c20932 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:09.197655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c7656-bc20-4c74-af35-01ffa029693e service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:09.208349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c7656-bc20-4c74-af35-01ffa029693e service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:09.270850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:09.271472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:09.271789 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:09.272061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] PATH_INFO: `/v3/users/bf0b8194fa644dfcafe152b12a163862` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:09.273438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf0b8194fa644dfcafe152b12a163862)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:09.275509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:09.343886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf0b8194fa644dfcafe152b12a163862'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:09.344766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf0b8194fa644dfcafe152b12a163862'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:09.354139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf0b8194fa644dfcafe152b12a163862'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:09.368569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:09.370287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf0b8194fa644dfcafe152b12a163862'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:09.379609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf0b8194fa644dfcafe152b12a163862'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:09.394944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1126/2243] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:55:09 2022] DELETE /identity/v3/users/bf0b8194fa644dfcafe152b12a163862 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:10.146406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:10.147805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:10.160562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b77c9-d651-4ff4-8360-9cdb95d86f2b admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:10.262708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:10.264259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:10.274442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:10.275571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:10.276224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:10.276570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] PATH_INFO: `/v3/projects/c0571a92c8bd4e25aa54c16083e57f45` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:10.279861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0571a92c8bd4e25aa54c16083e57f45)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:10.282177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:10.344762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:10.362135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6ba39-94c4-4eff-b6be-4fddcd58c2bb admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:10.362881 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0571a92c8bd4e25aa54c16083e57f45', 'request_id': 'req-f54aa661-ff7f-452d-a284-8c2ae789e74e'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:10.372812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0571a92c8bd4e25aa54c16083e57f45', 'request_id': 'req-f54aa661-ff7f-452d-a284-8c2ae789e74e'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:10.383110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0571a92c8bd4e25aa54c16083e57f45', 'request_id': 'req-f54aa661-ff7f-452d-a284-8c2ae789e74e'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:10.401517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] The token cache is being invalidate because project c0571a92c8bd4e25aa54c16083e57f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:55:10.402154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:10.403590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:10.409242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1118/2244] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:55:10 2022] DELETE /identity/v3/projects/c0571a92c8bd4e25aa54c16083e57f45 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:10.410373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:10.412834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:10.425900 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01b207-f58f-49f7-88b9-f23a108999eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:10.426526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01b207-f58f-49f7-88b9-f23a108999eb service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:10.427965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01b207-f58f-49f7-88b9-f23a108999eb service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:10.428262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01b207-f58f-49f7-88b9-f23a108999eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:10.430504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ef01b207-f58f-49f7-88b9-f23a108999eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:10.503730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aa661-ff7f-452d-a284-8c2ae789e74e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:10.509655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:10.510412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:10.510794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:10.511958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] PATH_INFO: `/v3/users/2f8107e4fa1948b38e8c69e7b2b18815` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:10.513318 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8107e4fa1948b38e8c69e7b2b18815)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:10.514957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01b207-f58f-49f7-88b9-f23a108999eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:10.515848 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:10.517022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01b207-f58f-49f7-88b9-f23a108999eb service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:10.529217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1127/2245] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:55:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:55:10.593314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8107e4fa1948b38e8c69e7b2b18815'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:10.594321 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8107e4fa1948b38e8c69e7b2b18815'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:10.606758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8107e4fa1948b38e8c69e7b2b18815'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:10.618462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8107e4fa1948b38e8c69e7b2b18815'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:10.620449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-cb54beb0-05d1-4943-909b-97548cc97993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8107e4fa1948b38e8c69e7b2b18815'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:10.647846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1119/2246] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:55:10 2022] DELETE /identity/v3/users/2f8107e4fa1948b38e8c69e7b2b18815 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:11.185041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01b207-f58f-49f7-88b9-f23a108999eb service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:11.197036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01b207-f58f-49f7-88b9-f23a108999eb service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:11.300589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:11.325986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:11.328197 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:11.328577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:11.328952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] PATH_INFO: `/v3/projects/0e29fd6872ac462cb9f9dda86f97ec5e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:11.330360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e29fd6872ac462cb9f9dda86f97ec5e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:11.332464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:11.428449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e29fd6872ac462cb9f9dda86f97ec5e', 'request_id': 'req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:11.440360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e29fd6872ac462cb9f9dda86f97ec5e', 'request_id': 'req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:11.449057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e29fd6872ac462cb9f9dda86f97ec5e', 'request_id': 'req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:11.463401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] The token cache is being invalidate because project 0e29fd6872ac462cb9f9dda86f97ec5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:55:11.463691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:11.466968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e5ae31e2-f8a0-4c44-96d0-920b2474cd8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:55:11.471387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1128/2247] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:55:11 2022] DELETE /identity/v3/projects/0e29fd6872ac462cb9f9dda86f97ec5e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:55:11.476248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4067b7-5063-4dcf-b733-db726ab9d2fc None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:11.476716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4067b7-5063-4dcf-b733-db726ab9d2fc None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:11.476809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4067b7-5063-4dcf-b733-db726ab9d2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:11.503945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ef4067b7-5063-4dcf-b733-db726ab9d2fc None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:11.505685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ef4067b7-5063-4dcf-b733-db726ab9d2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:11.554829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:11.580254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1120/2248] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:55:11.588251 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24947a63-4a5f-49b0-a91e-d305257ba0d5 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:11.588816 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24947a63-4a5f-49b0-a91e-d305257ba0d5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:11.589101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24947a63-4a5f-49b0-a91e-d305257ba0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:11.612074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-24947a63-4a5f-49b0-a91e-d305257ba0d5 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:11.613814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-24947a63-4a5f-49b0-a91e-d305257ba0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:11.625542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1129/2249] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:55:11.629864 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4067b7-5063-4dcf-b733-db726ab9d2fc None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:11.640766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4067b7-5063-4dcf-b733-db726ab9d2fc None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:11.646238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da535b1-4846-4b45-8d9c-357c100ee852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:11.646957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da535b1-4846-4b45-8d9c-357c100ee852 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:11.647311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da535b1-4846-4b45-8d9c-357c100ee852 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:11.647660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da535b1-4846-4b45-8d9c-357c100ee852 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:11.648396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da535b1-4846-4b45-8d9c-357c100ee852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:55:11.648774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da535b1-4846-4b45-8d9c-357c100ee852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:11.650511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da535b1-4846-4b45-8d9c-357c100ee852 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:11.669264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1121/2250] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:55:11.670673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24947a63-4a5f-49b0-a91e-d305257ba0d5 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:11.684658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24947a63-4a5f-49b0-a91e-d305257ba0d5 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:11.689628 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca5853-b373-4873-b12f-75bfb07292fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:11.690357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca5853-b373-4873-b12f-75bfb07292fb admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:11.690644 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca5853-b373-4873-b12f-75bfb07292fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:11.690914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca5853-b373-4873-b12f-75bfb07292fb admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:11.692443 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca5853-b373-4873-b12f-75bfb07292fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:11.695714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca5853-b373-4873-b12f-75bfb07292fb admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:11.734220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1130/2251] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:55:11 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:55:11.735665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da535b1-4846-4b45-8d9c-357c100ee852 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:11.745911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da535b1-4846-4b45-8d9c-357c100ee852 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:11.812771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a453a4-f3ea-45ed-9db2-8a2ffdf93770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:11.813961 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a453a4-f3ea-45ed-9db2-8a2ffdf93770 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:11.814499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a453a4-f3ea-45ed-9db2-8a2ffdf93770 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:11.814980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a453a4-f3ea-45ed-9db2-8a2ffdf93770 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:11.816090 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a453a4-f3ea-45ed-9db2-8a2ffdf93770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:11.818769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a453a4-f3ea-45ed-9db2-8a2ffdf93770 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:11.873267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1122/2252] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:55:11 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:55:11.874218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca5853-b373-4873-b12f-75bfb07292fb admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:11.891586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca5853-b373-4873-b12f-75bfb07292fb admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:11.907342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2a56a-a985-4f45-b8e5-1652d283bec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:11.908781 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a56a-a985-4f45-b8e5-1652d283bec8 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:11.909320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a56a-a985-4f45-b8e5-1652d283bec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:11.909611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a56a-a985-4f45-b8e5-1652d283bec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:11.910802 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2a56a-a985-4f45-b8e5-1652d283bec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:11.914288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2a56a-a985-4f45-b8e5-1652d283bec8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:11.934553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a453a4-f3ea-45ed-9db2-8a2ffdf93770 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:11.941718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1131/2253] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:55:11.946385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a453a4-f3ea-45ed-9db2-8a2ffdf93770 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:11.967689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd5702-7c52-4c80-a41b-7291877f0e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:11.968519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5702-7c52-4c80-a41b-7291877f0e9f admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:11.968907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5702-7c52-4c80-a41b-7291877f0e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:11.969305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5702-7c52-4c80-a41b-7291877f0e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:11.970277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd5702-7c52-4c80-a41b-7291877f0e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:11.972072 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd5702-7c52-4c80-a41b-7291877f0e9f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:11.990975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2a56a-a985-4f45-b8e5-1652d283bec8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:11.992449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1123/2254] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:55:12.003266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2a56a-a985-4f45-b8e5-1652d283bec8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:12.017796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f52a8-1f1c-4039-81d3-5907c0447e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:12.018668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f52a8-1f1c-4039-81d3-5907c0447e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:12.019053 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f52a8-1f1c-4039-81d3-5907c0447e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:12.019431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f52a8-1f1c-4039-81d3-5907c0447e0f admin admin] PATH_INFO: `/v3/projects/c3d2ddfbe27d4890bd703d6b99e9d2b4/users/91b1bae006ae42b5b8c325a4d2cefac0/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:12.065718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f52a8-1f1c-4039-81d3-5907c0447e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d2ddfbe27d4890bd703d6b99e9d2b4, user_id=91b1bae006ae42b5b8c325a4d2cefac0, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:12.069888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f52a8-1f1c-4039-81d3-5907c0447e0f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:12.103007 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1132/2255] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:55:11 2022] PUT /identity/v3/projects/c3d2ddfbe27d4890bd703d6b99e9d2b4/users/91b1bae006ae42b5b8c325a4d2cefac0/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:12.115674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840095ad-feee-4a31-8a7e-b6643bc338a8 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:12.116066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840095ad-feee-4a31-8a7e-b6643bc338a8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:12.116399 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840095ad-feee-4a31-8a7e-b6643bc338a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:12.163940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-840095ad-feee-4a31-8a7e-b6643bc338a8 None None] MFA Rules not processed for user `91b1bae006ae42b5b8c325a4d2cefac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:12.165727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-840095ad-feee-4a31-8a7e-b6643bc338a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:12.302806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1124/2256] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:55:12.311765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f52a8-1f1c-4039-81d3-5907c0447e0f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:12.314690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d75f52a8-1f1c-4039-81d3-5907c0447e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:12.437085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f52a8-1f1c-4039-81d3-5907c0447e0f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:12.441552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa26e7c-40e4-45c9-9bfb-17185dcbe1b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:12.442631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa26e7c-40e4-45c9-9bfb-17185dcbe1b3 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:12.443029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa26e7c-40e4-45c9-9bfb-17185dcbe1b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:12.443535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa26e7c-40e4-45c9-9bfb-17185dcbe1b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:12.478999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa26e7c-40e4-45c9-9bfb-17185dcbe1b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:12.481199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa26e7c-40e4-45c9-9bfb-17185dcbe1b3 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:12.493619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1133/2257] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:55:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:55:14.694719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590febe-80ef-4b06-9e82-9d95bdf6077a None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:14.694719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590febe-80ef-4b06-9e82-9d95bdf6077a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:14.695315 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590febe-80ef-4b06-9e82-9d95bdf6077a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:14.753176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-2590febe-80ef-4b06-9e82-9d95bdf6077a None None] MFA Rules not processed for user `7725c72e89964487843bc9bd46b8e3d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:14.754872 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2590febe-80ef-4b06-9e82-9d95bdf6077a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:14.791453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:14.834956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1125/2258] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:55:14.839860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa26e7c-40e4-45c9-9bfb-17185dcbe1b3 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:14.844498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-daa26e7c-40e4-45c9-9bfb-17185dcbe1b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:14.894299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:14.942961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa26e7c-40e4-45c9-9bfb-17185dcbe1b3 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:14.947705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f510-1912-4fb5-8b7d-0b2d6af12f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:14.948320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f510-1912-4fb5-8b7d-0b2d6af12f8a admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:14.948601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f510-1912-4fb5-8b7d-0b2d6af12f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:14.948873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f510-1912-4fb5-8b7d-0b2d6af12f8a admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:14.949589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194f510-1912-4fb5-8b7d-0b2d6af12f8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:14.951542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194f510-1912-4fb5-8b7d-0b2d6af12f8a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:14.984515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1134/2259] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:55:14 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:55:14.985311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590febe-80ef-4b06-9e82-9d95bdf6077a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:14.996605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590febe-80ef-4b06-9e82-9d95bdf6077a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:15.055955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f4cfc-581d-4149-a5e2-f8c80eed5515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:15.056579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f4cfc-581d-4149-a5e2-f8c80eed5515 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:15.056855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f4cfc-581d-4149-a5e2-f8c80eed5515 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:15.057120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f4cfc-581d-4149-a5e2-f8c80eed5515 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:15.057780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f4cfc-581d-4149-a5e2-f8c80eed5515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:15.059823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f4cfc-581d-4149-a5e2-f8c80eed5515 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:15.085792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1126/2260] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:55:14 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:55:15.090669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f510-1912-4fb5-8b7d-0b2d6af12f8a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:15.106117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f510-1912-4fb5-8b7d-0b2d6af12f8a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:15.111924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2003176-4107-41c9-80cd-e22f1e4ac719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:15.112248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2003176-4107-41c9-80cd-e22f1e4ac719 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:15.112555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2003176-4107-41c9-80cd-e22f1e4ac719 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:15.112878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2003176-4107-41c9-80cd-e22f1e4ac719 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:15.113767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2003176-4107-41c9-80cd-e22f1e4ac719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:15.115798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2003176-4107-41c9-80cd-e22f1e4ac719 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:15.128314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1135/2261] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:55:15.132568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f4cfc-581d-4149-a5e2-f8c80eed5515 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:15.143519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f4cfc-581d-4149-a5e2-f8c80eed5515 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:15.148254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4d0c7-225f-49a8-ac26-a95d2f6b5950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:15.149009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4d0c7-225f-49a8-ac26-a95d2f6b5950 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:15.149315 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4d0c7-225f-49a8-ac26-a95d2f6b5950 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:15.149628 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4d0c7-225f-49a8-ac26-a95d2f6b5950 admin admin] PATH_INFO: `/v3/projects/56b805f66853429593a3f0b37b907b3f/users/21b1d77c6794409e98576eb207a843cb/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:15.166368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4d0c7-225f-49a8-ac26-a95d2f6b5950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b805f66853429593a3f0b37b907b3f, user_id=21b1d77c6794409e98576eb207a843cb, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:15.168986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4d0c7-225f-49a8-ac26-a95d2f6b5950 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:15.183027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1127/2262] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:55:15 2022] PUT /identity/v3/projects/56b805f66853429593a3f0b37b907b3f/users/21b1d77c6794409e98576eb207a843cb/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:15.187399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2003176-4107-41c9-80cd-e22f1e4ac719 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:15.197708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2003176-4107-41c9-80cd-e22f1e4ac719 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:15.256301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531835c-f775-4ea1-a444-fafd719fa71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:15.257020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531835c-f775-4ea1-a444-fafd719fa71e admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:15.257295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531835c-f775-4ea1-a444-fafd719fa71e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:15.257604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531835c-f775-4ea1-a444-fafd719fa71e admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:15.258261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531835c-f775-4ea1-a444-fafd719fa71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:15.259888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531835c-f775-4ea1-a444-fafd719fa71e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:15.270822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1136/2263] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:55:15.275151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4d0c7-225f-49a8-ac26-a95d2f6b5950 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:15.292068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4d0c7-225f-49a8-ac26-a95d2f6b5950 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:15.297117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94760631-ea79-447f-92af-e9a715e7ed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:15.297999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94760631-ea79-447f-92af-e9a715e7ed76 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:15.298546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94760631-ea79-447f-92af-e9a715e7ed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:15.299779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94760631-ea79-447f-92af-e9a715e7ed76 admin admin] PATH_INFO: `/v3/projects/56b805f66853429593a3f0b37b907b3f/users/21b1d77c6794409e98576eb207a843cb/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:15.301044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94760631-ea79-447f-92af-e9a715e7ed76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b805f66853429593a3f0b37b907b3f, user_id=21b1d77c6794409e98576eb207a843cb, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:15.303761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94760631-ea79-447f-92af-e9a715e7ed76 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:15.314894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1128/2264] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:55:15 2022] PUT /identity/v3/projects/56b805f66853429593a3f0b37b907b3f/users/21b1d77c6794409e98576eb207a843cb/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:55:19.787313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f6e97-913a-4283-b322-313b94adf0a7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:19.787822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f6e97-913a-4283-b322-313b94adf0a7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:19.787927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f6e97-913a-4283-b322-313b94adf0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:19.820075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d46f6e97-913a-4283-b322-313b94adf0a7 None None] MFA Rules not processed for user `21b1d77c6794409e98576eb207a843cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:19.822408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d46f6e97-913a-4283-b322-313b94adf0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:19.923589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1137/2265] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:55:19.937674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94760631-ea79-447f-92af-e9a715e7ed76 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:55:19.939899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-94760631-ea79-447f-92af-e9a715e7ed76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:20.025847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94760631-ea79-447f-92af-e9a715e7ed76 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:55:20.032099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38b604-d93b-4ca2-9264-340a0986539f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:55:20.032836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38b604-d93b-4ca2-9264-340a0986539f service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:20.033128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38b604-d93b-4ca2-9264-340a0986539f service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:20.033401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38b604-d93b-4ca2-9264-340a0986539f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:20.049374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:20.101158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38b604-d93b-4ca2-9264-340a0986539f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:55:20.103433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38b604-d93b-4ca2-9264-340a0986539f service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:55:20.138630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1129/2266] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:55:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:55:21.248560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1fe17-f9c6-4169-88b4-5996b4ac9a23 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:21.249117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1fe17-f9c6-4169-88b4-5996b4ac9a23 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:21.249117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1fe17-f9c6-4169-88b4-5996b4ac9a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:21.282122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-3ac1fe17-f9c6-4169-88b4-5996b4ac9a23 None None] MFA Rules not processed for user `7725c72e89964487843bc9bd46b8e3d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:21.282122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3ac1fe17-f9c6-4169-88b4-5996b4ac9a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:21.296049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1138/2267] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:55:30.133590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=20 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:31.264469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=93,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:55:40.619360 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216485ca-5288-4f46-89b2-d64224cd0731 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:55:40.619845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216485ca-5288-4f46-89b2-d64224cd0731 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:55:40.619938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216485ca-5288-4f46-89b2-d64224cd0731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:55:40.640061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-216485ca-5288-4f46-89b2-d64224cd0731 None None] MFA Rules not processed for user `89c747f18b434e469ae61755e9f639ca`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:55:40.641326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-216485ca-5288-4f46-89b2-d64224cd0731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:55:40.698163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1130/2268] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:55:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:55:50.691358 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=25 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:56:11.022709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7436e2-2f59-4fb7-83c2-3badbe4ad549 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:11.023195 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7436e2-2f59-4fb7-83c2-3badbe4ad549 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:11.023290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7436e2-2f59-4fb7-83c2-3badbe4ad549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:11.043400 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-4e7436e2-2f59-4fb7-83c2-3badbe4ad549 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:56:11.044601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4e7436e2-2f59-4fb7-83c2-3badbe4ad549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:11.102212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1139/2269] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 14:56:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:56:11.151031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216485ca-5288-4f46-89b2-d64224cd0731 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:11.152472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-216485ca-5288-4f46-89b2-d64224cd0731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:11.222482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216485ca-5288-4f46-89b2-d64224cd0731 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:11.227397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb31850-96dd-485c-832d-a9143c617bf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:11.228050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb31850-96dd-485c-832d-a9143c617bf9 service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:11.228310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb31850-96dd-485c-832d-a9143c617bf9 service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:11.228580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb31850-96dd-485c-832d-a9143c617bf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:11.237097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb31850-96dd-485c-832d-a9143c617bf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:11.238724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb31850-96dd-485c-832d-a9143c617bf9 service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:11.247845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1131/2270] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 14:56:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 14:56:11.794475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7436e2-2f59-4fb7-83c2-3badbe4ad549 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:11.865028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7436e2-2f59-4fb7-83c2-3badbe4ad549 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:11.869817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:11.870504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:11.870942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:11.871284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] PATH_INFO: `/v3/users/91b1bae006ae42b5b8c325a4d2cefac0` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:11.872775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91b1bae006ae42b5b8c325a4d2cefac0)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:11.874909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:11.925113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91b1bae006ae42b5b8c325a4d2cefac0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:56:11.925893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91b1bae006ae42b5b8c325a4d2cefac0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:56:11.933361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91b1bae006ae42b5b8c325a4d2cefac0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:56:11.945427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91b1bae006ae42b5b8c325a4d2cefac0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:56:11.954333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91b1bae006ae42b5b8c325a4d2cefac0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:56:11.960743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1140/2271] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:56:11 2022] DELETE /identity/v3/users/91b1bae006ae42b5b8c325a4d2cefac0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:56:11.967048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a19df-1d75-475c-8f74-3ef094cc5d1a None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:11.967330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a19df-1d75-475c-8f74-3ef094cc5d1a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:11.967603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a19df-1d75-475c-8f74-3ef094cc5d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:11.989941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-111a19df-1d75-475c-8f74-3ef094cc5d1a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:56:11.991134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-111a19df-1d75-475c-8f74-3ef094cc5d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:12.086942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1132/2272] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:56:12.098298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:12.099743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:12.129868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:56:12.170906 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994d9fa-1132-4c02-bb2f-01994b750850 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:12.175354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477c7f-aaf0-459c-ba7e-7d256334c306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:12.175994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477c7f-aaf0-459c-ba7e-7d256334c306 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:12.176352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477c7f-aaf0-459c-ba7e-7d256334c306 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:12.176815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477c7f-aaf0-459c-ba7e-7d256334c306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:12.188739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477c7f-aaf0-459c-ba7e-7d256334c306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:12.190239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477c7f-aaf0-459c-ba7e-7d256334c306 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:12.200538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1141/2273] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:56:12.533536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a19df-1d75-475c-8f74-3ef094cc5d1a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:12.535438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-111a19df-1d75-475c-8f74-3ef094cc5d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:12.560368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a19df-1d75-475c-8f74-3ef094cc5d1a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:12.566745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:12.567544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:12.567889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:12.568230 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] PATH_INFO: `/v3/projects/c3d2ddfbe27d4890bd703d6b99e9d2b4` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:12.570980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d2ddfbe27d4890bd703d6b99e9d2b4)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:12.585550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:12.715597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d2ddfbe27d4890bd703d6b99e9d2b4', 'request_id': 'req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:56:12.731515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d2ddfbe27d4890bd703d6b99e9d2b4', 'request_id': 'req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:56:12.746955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d2ddfbe27d4890bd703d6b99e9d2b4', 'request_id': 'req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:56:12.770503 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] The token cache is being invalidate because project c3d2ddfbe27d4890bd703d6b99e9d2b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:56:12.770885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:56:12.771981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-9ae8df9f-e677-4a1e-af59-d8e10cfb4581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:56:12.774410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1133/2274] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:56:12 2022] DELETE /identity/v3/projects/c3d2ddfbe27d4890bd703d6b99e9d2b4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:56:20.440256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477c7f-aaf0-459c-ba7e-7d256334c306 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:20.442035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b5477c7f-aaf0-459c-ba7e-7d256334c306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:20.454204 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477c7f-aaf0-459c-ba7e-7d256334c306 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:20.510280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8bcc7-fb1a-4384-b384-3f75df63ae47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:20.510891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8bcc7-fb1a-4384-b384-3f75df63ae47 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:20.511685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8bcc7-fb1a-4384-b384-3f75df63ae47 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:20.511974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8bcc7-fb1a-4384-b384-3f75df63ae47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:20.513737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5ba8bcc7-fb1a-4384-b384-3f75df63ae47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:20.596285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8bcc7-fb1a-4384-b384-3f75df63ae47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:20.598500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8bcc7-fb1a-4384-b384-3f75df63ae47 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:20.612713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1142/2275] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:56:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:56:22.766425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:56:27.937286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31d5b9-2da4-4756-840d-26361f872716 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:27.937965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31d5b9-2da4-4756-840d-26361f872716 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:27.937965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31d5b9-2da4-4756-840d-26361f872716 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:27.939445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1134/2276] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:56:27.944968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1148f-1543-4f2b-9083-7b9d08fd641c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:27.945461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1148f-1543-4f2b-9083-7b9d08fd641c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:27.945731 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1148f-1543-4f2b-9083-7b9d08fd641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:27.965779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-97b1148f-1543-4f2b-9083-7b9d08fd641c None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:56:27.967022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-97b1148f-1543-4f2b-9083-7b9d08fd641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:28.038515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1143/2277] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:56:28.051271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31d5b9-2da4-4756-840d-26361f872716 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:28.052987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3f31d5b9-2da4-4756-840d-26361f872716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:28.114481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31d5b9-2da4-4756-840d-26361f872716 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:28.118944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58250f0f-2f77-4591-a763-9bd6cc24c28e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd04df68f9d84fc4bb981c8b69cc771a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:28.119567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58250f0f-2f77-4591-a763-9bd6cc24c28e service glance] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:28.119852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58250f0f-2f77-4591-a763-9bd6cc24c28e service glance] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:28.120731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58250f0f-2f77-4591-a763-9bd6cc24c28e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:28.131541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58250f0f-2f77-4591-a763-9bd6cc24c28e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:28.133006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58250f0f-2f77-4591-a763-9bd6cc24c28e service glance] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:28.144463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1135/2278] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:56:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:56:28.216071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3999c6-72f7-4043-9323-9ab773c37ac4 None None] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:28.216357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3999c6-72f7-4043-9323-9ab773c37ac4 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:28.216653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3999c6-72f7-4043-9323-9ab773c37ac4 None None] PATH_INFO: `/` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:28.218004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1144/2279] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:56:28.222959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353b215-364a-44b3-96a8-c807123f33c9 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:28.223247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353b215-364a-44b3-96a8-c807123f33c9 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:28.223519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353b215-364a-44b3-96a8-c807123f33c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:28.243418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f353b215-364a-44b3-96a8-c807123f33c9 None None] MFA Rules not processed for user `05e9009410a54ce3ab88fa0cac87a7f9`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:56:28.244623 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f353b215-364a-44b3-96a8-c807123f33c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:28.251419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1136/2280] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:56:28.263051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3999c6-72f7-4043-9323-9ab773c37ac4 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:28.264237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9f3999c6-72f7-4043-9323-9ab773c37ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:28.278314 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3999c6-72f7-4043-9323-9ab773c37ac4 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:28.288228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa155dd-12c3-4480-b5d8-1842985d482a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:28.288844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa155dd-12c3-4480-b5d8-1842985d482a service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:28.289124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa155dd-12c3-4480-b5d8-1842985d482a service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:28.289402 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa155dd-12c3-4480-b5d8-1842985d482a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:28.295785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:56:28.298208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa155dd-12c3-4480-b5d8-1842985d482a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:28.300208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa155dd-12c3-4480-b5d8-1842985d482a service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:28.308039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1145/2281] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:56:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:56:28.388321 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b45a8-63e0-4dc9-a70c-85475bdec3e8 None None] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:28.388605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b45a8-63e0-4dc9-a70c-85475bdec3e8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:28.388878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b45a8-63e0-4dc9-a70c-85475bdec3e8 None None] PATH_INFO: `/` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:28.390278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1137/2282] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 3 14:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 14:56:28.395653 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee2b7-8e33-4a90-9aec-4f260e9dfd97 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:28.395885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee2b7-8e33-4a90-9aec-4f260e9dfd97 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:28.396160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee2b7-8e33-4a90-9aec-4f260e9dfd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:28.416221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a76ee2b7-8e33-4a90-9aec-4f260e9dfd97 None None] MFA Rules not processed for user `1a4a95ae5ee84f259ffcd7528b0fa505`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:56:28.418399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a76ee2b7-8e33-4a90-9aec-4f260e9dfd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:28.427624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1146/2283] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 3 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:56:28.437931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b45a8-63e0-4dc9-a70c-85475bdec3e8 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:28.439595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1e4b45a8-63e0-4dc9-a70c-85475bdec3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:28.452541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b45a8-63e0-4dc9-a70c-85475bdec3e8 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:28.456098 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458d63d-6b48-4686-8064-312ba8de8100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:28.456684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458d63d-6b48-4686-8064-312ba8de8100 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:28.456959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458d63d-6b48-4686-8064-312ba8de8100 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:28.457227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458d63d-6b48-4686-8064-312ba8de8100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:28.467779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458d63d-6b48-4686-8064-312ba8de8100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:28.469199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458d63d-6b48-4686-8064-312ba8de8100 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:28.480228 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1138/2284] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:56:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:56:31.718199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49e8d-99e4-42ba-9c1e-185504b2cdd4 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:31.718689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49e8d-99e4-42ba-9c1e-185504b2cdd4 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:31.718784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49e8d-99e4-42ba-9c1e-185504b2cdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:31.739591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-3bf49e8d-99e4-42ba-9c1e-185504b2cdd4 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:56:31.740788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3bf49e8d-99e4-42ba-9c1e-185504b2cdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:31.795650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1147/2285] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 14:56:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:56:31.808992 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458d63d-6b48-4686-8064-312ba8de8100 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:31.810611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d458d63d-6b48-4686-8064-312ba8de8100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:31.878912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458d63d-6b48-4686-8064-312ba8de8100 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:31.883861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fc26f-9e80-4ac1-9fb7-b19c142b4f68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:31.884459 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fc26f-9e80-4ac1-9fb7-b19c142b4f68 service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:31.884736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fc26f-9e80-4ac1-9fb7-b19c142b4f68 service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:31.885003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fc26f-9e80-4ac1-9fb7-b19c142b4f68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:31.893579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fc26f-9e80-4ac1-9fb7-b19c142b4f68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:31.895103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fc26f-9e80-4ac1-9fb7-b19c142b4f68 service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:31.904202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1139/2286] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 14:56:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 14:56:41.788385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=30 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:56:41.900152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=67 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:56:49.524528 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf49e8d-99e4-42ba-9c1e-185504b2cdd4 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:49.534419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf49e8d-99e4-42ba-9c1e-185504b2cdd4 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:49.538115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f180b0-6618-4583-8d54-9907be113bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:49.538718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f180b0-6618-4583-8d54-9907be113bed service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:49.538999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f180b0-6618-4583-8d54-9907be113bed service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:49.539302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f180b0-6618-4583-8d54-9907be113bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:49.541091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b8f180b0-6618-4583-8d54-9907be113bed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:49.554039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f180b0-6618-4583-8d54-9907be113bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:49.555452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f180b0-6618-4583-8d54-9907be113bed service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:49.566077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1148/2287] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:56:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:56:50.598020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fc26f-9e80-4ac1-9fb7-b19c142b4f68 service swift] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:56:50.607349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fc26f-9e80-4ac1-9fb7-b19c142b4f68 service swift] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:56:50.611566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9061a7-0c6c-482b-b1cd-cd7ec6d18521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:56:50.612154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9061a7-0c6c-482b-b1cd-cd7ec6d18521 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:56:50.612435 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9061a7-0c6c-482b-b1cd-cd7ec6d18521 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:56:50.612704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9061a7-0c6c-482b-b1cd-cd7ec6d18521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:56:50.614524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8e9061a7-0c6c-482b-b1cd-cd7ec6d18521 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:56:50.627618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9061a7-0c6c-482b-b1cd-cd7ec6d18521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:56:50.629034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9061a7-0c6c-482b-b1cd-cd7ec6d18521 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:56:50.639771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1140/2288] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:56:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:56:59.561304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=6 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:57:00.635014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=6 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:57:02.258517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e140f-b47c-49b0-bce7-2d41fe1d7ec8 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:57:02.258986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e140f-b47c-49b0-bce7-2d41fe1d7ec8 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:57:02.259082 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e140f-b47c-49b0-bce7-2d41fe1d7ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:57:02.277843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-244e140f-b47c-49b0-bce7-2d41fe1d7ec8 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:57:02.279057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-244e140f-b47c-49b0-bce7-2d41fe1d7ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:02.286556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1149/2289] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 3 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:57:02.299810 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9061a7-0c6c-482b-b1cd-cd7ec6d18521 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:57:02.310941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9061a7-0c6c-482b-b1cd-cd7ec6d18521 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:57:02.316642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844302-ce7c-4931-a02b-5a1411a5e5de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:57:02.317267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844302-ce7c-4931-a02b-5a1411a5e5de service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:57:02.317579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844302-ce7c-4931-a02b-5a1411a5e5de service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:57:02.317856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844302-ce7c-4931-a02b-5a1411a5e5de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:57:02.326484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0844302-ce7c-4931-a02b-5a1411a5e5de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:57:02.328071 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0844302-ce7c-4931-a02b-5a1411a5e5de service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:57:02.336291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1141/2290] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 14:57:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 14:57:09.585805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e140f-b47c-49b0-bce7-2d41fe1d7ec8 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:57:09.587494 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-244e140f-b47c-49b0-bce7-2d41fe1d7ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:09.603336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e140f-b47c-49b0-bce7-2d41fe1d7ec8 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:57:09.608661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028bdb2-4a74-4076-b6c5-6f5db5ecb5c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:57:09.609273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028bdb2-4a74-4076-b6c5-6f5db5ecb5c7 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:57:09.609579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028bdb2-4a74-4076-b6c5-6f5db5ecb5c7 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:57:09.609854 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028bdb2-4a74-4076-b6c5-6f5db5ecb5c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:57:09.611483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5028bdb2-4a74-4076-b6c5-6f5db5ecb5c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:09.667776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028bdb2-4a74-4076-b6c5-6f5db5ecb5c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:57:09.669359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028bdb2-4a74-4076-b6c5-6f5db5ecb5c7 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:57:09.678158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1150/2291] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:57:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:57:12.332947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=6 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:57:19.674085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=31 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:57:31.152066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844302-ce7c-4931-a02b-5a1411a5e5de service swift] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:57:31.161733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844302-ce7c-4931-a02b-5a1411a5e5de service swift] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:57:31.165462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b80cd5-bf0f-4267-9b21-ecbc5c4e08da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:57:31.166199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b80cd5-bf0f-4267-9b21-ecbc5c4e08da service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:57:31.166478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b80cd5-bf0f-4267-9b21-ecbc5c4e08da service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:57:31.166803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b80cd5-bf0f-4267-9b21-ecbc5c4e08da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:57:31.168490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-87b80cd5-bf0f-4267-9b21-ecbc5c4e08da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:31.178694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b80cd5-bf0f-4267-9b21-ecbc5c4e08da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:57:31.180115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b80cd5-bf0f-4267-9b21-ecbc5c4e08da service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:57:31.187897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1142/2292] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:57:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:57:33.255453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626eaa7-4952-49cd-90fb-eaf3839e5e23 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:57:33.256001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626eaa7-4952-49cd-90fb-eaf3839e5e23 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:57:33.256001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626eaa7-4952-49cd-90fb-eaf3839e5e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:57:33.274480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b626eaa7-4952-49cd-90fb-eaf3839e5e23 None None] MFA Rules not processed for user `335dd8706d4c402e98605479b57d059a`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:57:33.275666 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b626eaa7-4952-49cd-90fb-eaf3839e5e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:33.282990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1151/2293] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:57:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:57:39.558275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b80cd5-bf0f-4267-9b21-ecbc5c4e08da service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:57:39.571564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b80cd5-bf0f-4267-9b21-ecbc5c4e08da service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:57:39.577097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d088f56-8b75-4ec2-8078-de03a676d7a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:57:39.577724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d088f56-8b75-4ec2-8078-de03a676d7a8 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:57:39.578001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d088f56-8b75-4ec2-8078-de03a676d7a8 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:57:39.578270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d088f56-8b75-4ec2-8078-de03a676d7a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:57:39.580488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3d088f56-8b75-4ec2-8078-de03a676d7a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:39.592663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d088f56-8b75-4ec2-8078-de03a676d7a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:57:39.595021 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d088f56-8b75-4ec2-8078-de03a676d7a8 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:57:39.605380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1143/2294] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:57:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:57:42.794819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626eaa7-4952-49cd-90fb-eaf3839e5e23 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:57:42.807741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626eaa7-4952-49cd-90fb-eaf3839e5e23 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:57:42.813740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb0a62-a026-42bf-ade3-e2f8e926af08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:57:42.814335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb0a62-a026-42bf-ade3-e2f8e926af08 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:57:42.814610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb0a62-a026-42bf-ade3-e2f8e926af08 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:57:42.814878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb0a62-a026-42bf-ade3-e2f8e926af08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:57:42.816571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-15cb0a62-a026-42bf-ade3-e2f8e926af08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:42.872329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb0a62-a026-42bf-ade3-e2f8e926af08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:57:42.873963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb0a62-a026-42bf-ade3-e2f8e926af08 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:57:42.883303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1152/2295] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:57:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:57:49.601056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=12 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:57:52.879240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=31 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:57:56.501311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fcea9-c14c-46b7-b8be-941b81128a31 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:57:56.501817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fcea9-c14c-46b7-b8be-941b81128a31 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:57:56.501918 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fcea9-c14c-46b7-b8be-941b81128a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:57:56.520477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c27fcea9-c14c-46b7-b8be-941b81128a31 None None] MFA Rules not processed for user `89c747f18b434e469ae61755e9f639ca`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:57:56.521703 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c27fcea9-c14c-46b7-b8be-941b81128a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:56.528474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1144/2296] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:57:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:57:56.827535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb0a62-a026-42bf-ade3-e2f8e926af08 service nova] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:57:56.837233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb0a62-a026-42bf-ade3-e2f8e926af08 service nova] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:57:56.840787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56924c4a-8dbc-428a-857d-224a96933e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:57:56.841381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56924c4a-8dbc-428a-857d-224a96933e6f service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:57:56.841695 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56924c4a-8dbc-428a-857d-224a96933e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:57:56.841967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56924c4a-8dbc-428a-857d-224a96933e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:57:56.844292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-56924c4a-8dbc-428a-857d-224a96933e6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:56.853344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56924c4a-8dbc-428a-857d-224a96933e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:57:56.854782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56924c4a-8dbc-428a-857d-224a96933e6f service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:57:56.862337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1153/2297] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:57:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:57:57.170665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fcea9-c14c-46b7-b8be-941b81128a31 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:57:57.172191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c27fcea9-c14c-46b7-b8be-941b81128a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:57.243863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fcea9-c14c-46b7-b8be-941b81128a31 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:57:57.248890 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cf329-34bd-4015-8954-0df9848b317f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:57:57.249470 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cf329-34bd-4015-8954-0df9848b317f service placement] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:57:57.250283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cf329-34bd-4015-8954-0df9848b317f service placement] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:57:57.250551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cf329-34bd-4015-8954-0df9848b317f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:57:57.252218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-137cf329-34bd-4015-8954-0df9848b317f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:57:57.263091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cf329-34bd-4015-8954-0df9848b317f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:57:57.264649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cf329-34bd-4015-8954-0df9848b317f service placement] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:57:57.275146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1145/2298] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:57:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:58:06.859332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=6 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:58:07.270746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=35 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:58:07.850620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56924c4a-8dbc-428a-857d-224a96933e6f service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:58:07.864003 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56924c4a-8dbc-428a-857d-224a96933e6f service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:58:07.870036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d64e4-2409-4a59-b28a-35a0228668fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:58:07.870665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d64e4-2409-4a59-b28a-35a0228668fc service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:58:07.870947 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d64e4-2409-4a59-b28a-35a0228668fc service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:58:07.871219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d64e4-2409-4a59-b28a-35a0228668fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:58:07.872964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3e3d64e4-2409-4a59-b28a-35a0228668fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:58:07.885766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d64e4-2409-4a59-b28a-35a0228668fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:58:07.887367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d64e4-2409-4a59-b28a-35a0228668fc service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:58:07.898565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1154/2299] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:58:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:58:09.517695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cf329-34bd-4015-8954-0df9848b317f service placement] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:58:09.529747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cf329-34bd-4015-8954-0df9848b317f service placement] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:58:09.535183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181bca47-df8d-41a5-ae87-fbdfdd9ece63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:58:09.535771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bca47-df8d-41a5-ae87-fbdfdd9ece63 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:58:09.536048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bca47-df8d-41a5-ae87-fbdfdd9ece63 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:58:09.536316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bca47-df8d-41a5-ae87-fbdfdd9ece63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:58:09.538025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-181bca47-df8d-41a5-ae87-fbdfdd9ece63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:58:09.594853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181bca47-df8d-41a5-ae87-fbdfdd9ece63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:58:09.596444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181bca47-df8d-41a5-ae87-fbdfdd9ece63 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:58:09.605270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1146/2300] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:58:12.714546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f762e4-83be-40d5-84cc-82e7cad97550 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:58:12.715133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f762e4-83be-40d5-84cc-82e7cad97550 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:58:12.715133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f762e4-83be-40d5-84cc-82e7cad97550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:58:12.734946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f7f762e4-83be-40d5-84cc-82e7cad97550 None None] MFA Rules not processed for user `7cfae014351641d8a949775875a45958`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:58:12.736242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f7f762e4-83be-40d5-84cc-82e7cad97550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:58:12.797757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1155/2301] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:58:13.325129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181bca47-df8d-41a5-ae87-fbdfdd9ece63 service nova] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:58:13.341238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181bca47-df8d-41a5-ae87-fbdfdd9ece63 service nova] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:58:13.346615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c019c-da7f-4013-b1db-0ace59d3b2dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:58:13.347266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c019c-da7f-4013-b1db-0ace59d3b2dd service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:58:13.347656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c019c-da7f-4013-b1db-0ace59d3b2dd service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:58:13.348104 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c019c-da7f-4013-b1db-0ace59d3b2dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:58:13.350015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-962c019c-da7f-4013-b1db-0ace59d3b2dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:58:13.364215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c019c-da7f-4013-b1db-0ace59d3b2dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:58:13.365818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c019c-da7f-4013-b1db-0ace59d3b2dd service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:58:13.374962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1147/2302] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:58:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:58:20.411016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f762e4-83be-40d5-84cc-82e7cad97550 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:58:20.412554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f7f762e4-83be-40d5-84cc-82e7cad97550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:58:20.428482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f762e4-83be-40d5-84cc-82e7cad97550 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:58:20.434035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e12383-0d2a-4eb2-8ee7-60026cfa82f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:58:20.434657 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e12383-0d2a-4eb2-8ee7-60026cfa82f8 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:58:20.434938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e12383-0d2a-4eb2-8ee7-60026cfa82f8 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:58:20.435210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e12383-0d2a-4eb2-8ee7-60026cfa82f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:58:20.437455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c3e12383-0d2a-4eb2-8ee7-60026cfa82f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:58:20.496705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e12383-0d2a-4eb2-8ee7-60026cfa82f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:58:20.498964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e12383-0d2a-4eb2-8ee7-60026cfa82f8 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:58:20.508578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1156/2303] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:58:23.370446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=34 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:58:27.466471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c019c-da7f-4013-b1db-0ace59d3b2dd service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:58:27.481761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c019c-da7f-4013-b1db-0ace59d3b2dd service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:58:27.490866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b71731-a5e0-4f79-a334-84f7bafee584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:58:27.492069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b71731-a5e0-4f79-a334-84f7bafee584 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:58:27.492385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b71731-a5e0-4f79-a334-84f7bafee584 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:58:27.492669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b71731-a5e0-4f79-a334-84f7bafee584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:58:27.494475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-58b71731-a5e0-4f79-a334-84f7bafee584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:58:27.509665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b71731-a5e0-4f79-a334-84f7bafee584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:58:27.511311 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b71731-a5e0-4f79-a334-84f7bafee584 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:58:27.530402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1148/2304] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:58:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:58:30.503883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=59 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:58:31.233020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef474d0d-9435-40ca-8f95-5152cd9e8404 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:58:31.233312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef474d0d-9435-40ca-8f95-5152cd9e8404 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:58:31.233669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef474d0d-9435-40ca-8f95-5152cd9e8404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:58:31.253720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ef474d0d-9435-40ca-8f95-5152cd9e8404 None None] MFA Rules not processed for user `9e5325f010ae4359bd6b514a1995614d`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:58:31.254914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ef474d0d-9435-40ca-8f95-5152cd9e8404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:58:31.262658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1157/2305] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:58:37.526049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=6 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:58:41.248167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:06.976032 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b71731-a5e0-4f79-a334-84f7bafee584 service nova] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:06.985308 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b71731-a5e0-4f79-a334-84f7bafee584 service nova] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:06.988838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8495dc-417c-4543-a824-f4ff05a1995b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:06.989424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8495dc-417c-4543-a824-f4ff05a1995b service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:06.989808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8495dc-417c-4543-a824-f4ff05a1995b service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:06.990404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8495dc-417c-4543-a824-f4ff05a1995b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:06.992582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ec8495dc-417c-4543-a824-f4ff05a1995b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:07.055825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8495dc-417c-4543-a824-f4ff05a1995b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:07.057241 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8495dc-417c-4543-a824-f4ff05a1995b service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:07.067202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1149/2306] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:10.355756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef474d0d-9435-40ca-8f95-5152cd9e8404 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:10.355756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ef474d0d-9435-40ca-8f95-5152cd9e8404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:10.376237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef474d0d-9435-40ca-8f95-5152cd9e8404 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:10.380823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:10.381453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:10.381763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:10.382035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] PATH_INFO: `/v3/users/335dd8706d4c402e98605479b57d059a` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:10.383155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=335dd8706d4c402e98605479b57d059a)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:10.385081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:10.431790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '335dd8706d4c402e98605479b57d059a'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:10.432441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '335dd8706d4c402e98605479b57d059a'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:10.439449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '335dd8706d4c402e98605479b57d059a'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:10.451629 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '335dd8706d4c402e98605479b57d059a'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:10.459736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0ecefc77-e452-4905-9975-32ed74704a57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '335dd8706d4c402e98605479b57d059a'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:10.465750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1158/2307] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:10 2022] DELETE /identity/v3/users/335dd8706d4c402e98605479b57d059a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:10.645006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8495dc-417c-4543-a824-f4ff05a1995b service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:10.660925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8495dc-417c-4543-a824-f4ff05a1995b service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:10.718091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:10.718740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:10.719000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:10.719275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] PATH_INFO: `/v3/projects/83b495f3cb9b437d9572f4d33327fbf7` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:10.720442 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b495f3cb9b437d9572f4d33327fbf7)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:10.722345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:10.767780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b495f3cb9b437d9572f4d33327fbf7', 'request_id': 'req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:10.774584 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b495f3cb9b437d9572f4d33327fbf7', 'request_id': 'req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:10.781717 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b495f3cb9b437d9572f4d33327fbf7', 'request_id': 'req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:10.791664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] The token cache is being invalidate because project 83b495f3cb9b437d9572f4d33327fbf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:10.791949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:10.792565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-05e5f5a3-3c14-4161-8ae0-547eaf42ebce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:10.794057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1150/2308] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:10 2022] DELETE /identity/v3/projects/83b495f3cb9b437d9572f4d33327fbf7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:10.799605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd578c-155c-48d7-b590-1ac3e3b69e36 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:10.799891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd578c-155c-48d7-b590-1ac3e3b69e36 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:10.800182 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd578c-155c-48d7-b590-1ac3e3b69e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:10.819232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-92bd578c-155c-48d7-b590-1ac3e3b69e36 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:10.820424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-92bd578c-155c-48d7-b590-1ac3e3b69e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:10.879104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1159/2309] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:59:10.886943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158564b-abf7-4417-9c69-0dcf6ce9f770 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:10.887227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158564b-abf7-4417-9c69-0dcf6ce9f770 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:10.887528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158564b-abf7-4417-9c69-0dcf6ce9f770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:10.908519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-6158564b-abf7-4417-9c69-0dcf6ce9f770 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:10.909744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6158564b-abf7-4417-9c69-0dcf6ce9f770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:10.917389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1151/2310] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:10.922133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd578c-155c-48d7-b590-1ac3e3b69e36 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:10.935117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd578c-155c-48d7-b590-1ac3e3b69e36 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:10.941526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d6c95-dd5b-424c-a81d-43ce58a52c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:10.942386 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d6c95-dd5b-424c-a81d-43ce58a52c16 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:10.942795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d6c95-dd5b-424c-a81d-43ce58a52c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:10.943190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d6c95-dd5b-424c-a81d-43ce58a52c16 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:10.944153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d6c95-dd5b-424c-a81d-43ce58a52c16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:59:10.944609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d6c95-dd5b-424c-a81d-43ce58a52c16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:10.947112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d6c95-dd5b-424c-a81d-43ce58a52c16 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:10.962999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1160/2311] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:59:10.968070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158564b-abf7-4417-9c69-0dcf6ce9f770 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:10.977360 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158564b-abf7-4417-9c69-0dcf6ce9f770 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:10.981932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f4401-c0c3-4b46-9a5e-23177d9a3683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:10.982519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f4401-c0c3-4b46-9a5e-23177d9a3683 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:10.982795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f4401-c0c3-4b46-9a5e-23177d9a3683 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:10.983062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f4401-c0c3-4b46-9a5e-23177d9a3683 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:10.983698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f4401-c0c3-4b46-9a5e-23177d9a3683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:10.985641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f4401-c0c3-4b46-9a5e-23177d9a3683 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:11.000450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:11.015347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1152/2312] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:10 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:11.019497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d6c95-dd5b-424c-a81d-43ce58a52c16 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:11.030207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d6c95-dd5b-424c-a81d-43ce58a52c16 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:11.086758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af84ea1-40ca-4877-805b-2763b4798d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:11.087349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af84ea1-40ca-4877-805b-2763b4798d69 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:11.087627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af84ea1-40ca-4877-805b-2763b4798d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:11.087897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af84ea1-40ca-4877-805b-2763b4798d69 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:11.088532 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af84ea1-40ca-4877-805b-2763b4798d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:11.090492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af84ea1-40ca-4877-805b-2763b4798d69 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:11.098934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:11.114063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1161/2313] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:11 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:59:11.118127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f4401-c0c3-4b46-9a5e-23177d9a3683 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:11.128353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f4401-c0c3-4b46-9a5e-23177d9a3683 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:11.133891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d09f2-47f1-47b1-bbb1-256a3e1dd129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:11.134475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d09f2-47f1-47b1-bbb1-256a3e1dd129 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:11.134747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d09f2-47f1-47b1-bbb1-256a3e1dd129 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:11.135009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d09f2-47f1-47b1-bbb1-256a3e1dd129 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:11.135637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d09f2-47f1-47b1-bbb1-256a3e1dd129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:11.137225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d09f2-47f1-47b1-bbb1-256a3e1dd129 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:11.148148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1153/2314] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:11.151910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af84ea1-40ca-4877-805b-2763b4798d69 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:11.162460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af84ea1-40ca-4877-805b-2763b4798d69 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:11.167213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867d5c8-d7f9-49f5-81c4-98f8983ecf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:11.167791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867d5c8-d7f9-49f5-81c4-98f8983ecf3e admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:11.168071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867d5c8-d7f9-49f5-81c4-98f8983ecf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:11.168343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867d5c8-d7f9-49f5-81c4-98f8983ecf3e admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:11.168987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867d5c8-d7f9-49f5-81c4-98f8983ecf3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:11.170787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867d5c8-d7f9-49f5-81c4-98f8983ecf3e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:11.181206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1162/2315] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:11.185357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d09f2-47f1-47b1-bbb1-256a3e1dd129 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:11.195153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d09f2-47f1-47b1-bbb1-256a3e1dd129 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:11.199790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f43fa-d058-4d44-95b8-bc7b25ca188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:11.200425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f43fa-d058-4d44-95b8-bc7b25ca188a admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:11.200704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f43fa-d058-4d44-95b8-bc7b25ca188a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:11.200976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f43fa-d058-4d44-95b8-bc7b25ca188a admin admin] PATH_INFO: `/v3/projects/f9db3d362e9e4784a83cc3cd6fb6ddd5/users/2571ccc832ba4908954da476d6b267fb/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:11.215760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f43fa-d058-4d44-95b8-bc7b25ca188a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9db3d362e9e4784a83cc3cd6fb6ddd5, user_id=2571ccc832ba4908954da476d6b267fb, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:11.219060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f43fa-d058-4d44-95b8-bc7b25ca188a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:11.231397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1154/2316] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:11 2022] PUT /identity/v3/projects/f9db3d362e9e4784a83cc3cd6fb6ddd5/users/2571ccc832ba4908954da476d6b267fb/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:11.242179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39806cd0-576c-4194-ab4a-1be8dc7de977 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:11.242464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39806cd0-576c-4194-ab4a-1be8dc7de977 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:11.242738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39806cd0-576c-4194-ab4a-1be8dc7de977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:11.272858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-39806cd0-576c-4194-ab4a-1be8dc7de977 None None] MFA Rules not processed for user `2571ccc832ba4908954da476d6b267fb`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:11.274104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-39806cd0-576c-4194-ab4a-1be8dc7de977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:11.340022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1163/2317] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:59:11.351484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f43fa-d058-4d44-95b8-bc7b25ca188a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:11.353153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d17f43fa-d058-4d44-95b8-bc7b25ca188a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:11.426661 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f43fa-d058-4d44-95b8-bc7b25ca188a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:11.432375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e41ac-af6b-4324-a47e-992d5e9dbf39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:11.432988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e41ac-af6b-4324-a47e-992d5e9dbf39 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:11.433270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e41ac-af6b-4324-a47e-992d5e9dbf39 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:11.433570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e41ac-af6b-4324-a47e-992d5e9dbf39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:11.441993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e41ac-af6b-4324-a47e-992d5e9dbf39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:11.443591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e41ac-af6b-4324-a47e-992d5e9dbf39 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:11.453207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1155/2318] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:14.114534 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39806cd0-576c-4194-ab4a-1be8dc7de977 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:14.173982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39806cd0-576c-4194-ab4a-1be8dc7de977 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:14.178577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:14.179199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:14.179475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:14.179743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] PATH_INFO: `/v3/users/2571ccc832ba4908954da476d6b267fb` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:14.180835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2571ccc832ba4908954da476d6b267fb)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:14.182791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:14.227586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2571ccc832ba4908954da476d6b267fb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:14.228467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2571ccc832ba4908954da476d6b267fb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:14.235347 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2571ccc832ba4908954da476d6b267fb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:14.245372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:14.247962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2571ccc832ba4908954da476d6b267fb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:14.255881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2571ccc832ba4908954da476d6b267fb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:14.262204 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1164/2319] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:14 2022] DELETE /identity/v3/users/2571ccc832ba4908954da476d6b267fb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:14.267312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222f0ca-7798-444e-97a0-121d55354c76 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:14.267781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222f0ca-7798-444e-97a0-121d55354c76 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:14.267876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222f0ca-7798-444e-97a0-121d55354c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:14.287571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f222f0ca-7798-444e-97a0-121d55354c76 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:14.288771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f222f0ca-7798-444e-97a0-121d55354c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:14.347877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1156/2320] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:14.357559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:14.358944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:14.417292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ad629-87af-4522-ab74-07110aad70e4 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:14.422141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17541144-a29b-46d5-9f8c-f472458bf291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:14.422777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17541144-a29b-46d5-9f8c-f472458bf291 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:14.423059 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17541144-a29b-46d5-9f8c-f472458bf291 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:14.423336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17541144-a29b-46d5-9f8c-f472458bf291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:14.432868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17541144-a29b-46d5-9f8c-f472458bf291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:14.434328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17541144-a29b-46d5-9f8c-f472458bf291 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:14.444134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1165/2321] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:14.718483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222f0ca-7798-444e-97a0-121d55354c76 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:14.720482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f222f0ca-7798-444e-97a0-121d55354c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:14.753989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222f0ca-7798-444e-97a0-121d55354c76 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:14.761525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:14.762699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:14.763269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:14.763833 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] PATH_INFO: `/v3/projects/f9db3d362e9e4784a83cc3cd6fb6ddd5` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:14.765847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9db3d362e9e4784a83cc3cd6fb6ddd5)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:14.769547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:14.829197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:14.836839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9db3d362e9e4784a83cc3cd6fb6ddd5', 'request_id': 'req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:14.843965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9db3d362e9e4784a83cc3cd6fb6ddd5', 'request_id': 'req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:14.850110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9db3d362e9e4784a83cc3cd6fb6ddd5', 'request_id': 'req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:14.860845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] The token cache is being invalidate because project f9db3d362e9e4784a83cc3cd6fb6ddd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:14.861133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:14.861802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c2f3b3ab-b090-4efe-a6a9-9ebc2e34d22f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:14.863406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1157/2322] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:14 2022] DELETE /identity/v3/projects/f9db3d362e9e4784a83cc3cd6fb6ddd5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:14.868296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f50654-1ec4-40f0-861b-bbed0b2f9d50 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:14.868591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f50654-1ec4-40f0-861b-bbed0b2f9d50 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:14.868865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f50654-1ec4-40f0-861b-bbed0b2f9d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:14.889980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-c2f50654-1ec4-40f0-861b-bbed0b2f9d50 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:14.891183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c2f50654-1ec4-40f0-861b-bbed0b2f9d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:14.954785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1166/2323] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:14.962584 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dc872-d630-4359-a42e-b6474561e9af None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:14.962867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dc872-d630-4359-a42e-b6474561e9af None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:14.963137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dc872-d630-4359-a42e-b6474561e9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:14.985022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-847dc872-d630-4359-a42e-b6474561e9af None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:14.986266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-847dc872-d630-4359-a42e-b6474561e9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:14.999056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1158/2324] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:59:15.004392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f50654-1ec4-40f0-861b-bbed0b2f9d50 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.016248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f50654-1ec4-40f0-861b-bbed0b2f9d50 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.022233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40182e2-0377-470f-a8cb-c774a010578a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.023127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40182e2-0377-470f-a8cb-c774a010578a admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.023544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40182e2-0377-470f-a8cb-c774a010578a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.023881 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40182e2-0377-470f-a8cb-c774a010578a admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.024647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40182e2-0377-470f-a8cb-c774a010578a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:59:15.025013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40182e2-0377-470f-a8cb-c774a010578a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.026958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40182e2-0377-470f-a8cb-c774a010578a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.039908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1167/2325] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:59:15.043958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dc872-d630-4359-a42e-b6474561e9af None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.054695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dc872-d630-4359-a42e-b6474561e9af None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.059383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181815ed-afd0-4e9f-b64c-109ff8584d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.059985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181815ed-afd0-4e9f-b64c-109ff8584d87 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.060325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181815ed-afd0-4e9f-b64c-109ff8584d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.061086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181815ed-afd0-4e9f-b64c-109ff8584d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.061940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181815ed-afd0-4e9f-b64c-109ff8584d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.063839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181815ed-afd0-4e9f-b64c-109ff8584d87 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.092378 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1159/2326] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:15 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:15.096433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40182e2-0377-470f-a8cb-c774a010578a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.106819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40182e2-0377-470f-a8cb-c774a010578a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.166876 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9726389-bf9c-4f54-8171-c2d91cafd99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.167524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9726389-bf9c-4f54-8171-c2d91cafd99c admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.167758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9726389-bf9c-4f54-8171-c2d91cafd99c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.168033 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9726389-bf9c-4f54-8171-c2d91cafd99c admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.168679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9726389-bf9c-4f54-8171-c2d91cafd99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.171469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9726389-bf9c-4f54-8171-c2d91cafd99c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.195479 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1168/2327] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:15 2022] POST /identity/v3/users => generated 279 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:59:15.200662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181815ed-afd0-4e9f-b64c-109ff8584d87 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.211494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181815ed-afd0-4e9f-b64c-109ff8584d87 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.220590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad32d73-7150-40e7-a6d3-a205c81d8734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.221348 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad32d73-7150-40e7-a6d3-a205c81d8734 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.221753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad32d73-7150-40e7-a6d3-a205c81d8734 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.222119 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad32d73-7150-40e7-a6d3-a205c81d8734 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.223037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad32d73-7150-40e7-a6d3-a205c81d8734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.225326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad32d73-7150-40e7-a6d3-a205c81d8734 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.244752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1160/2328] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:15.246110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9726389-bf9c-4f54-8171-c2d91cafd99c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.263280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9726389-bf9c-4f54-8171-c2d91cafd99c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.268746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae94e6a-1762-4c2e-92bf-39d59940634a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.269377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae94e6a-1762-4c2e-92bf-39d59940634a admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.269705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae94e6a-1762-4c2e-92bf-39d59940634a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.270065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae94e6a-1762-4c2e-92bf-39d59940634a admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.270595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae94e6a-1762-4c2e-92bf-39d59940634a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.272177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae94e6a-1762-4c2e-92bf-39d59940634a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.274708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:15.302918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1169/2329] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:59:15.307184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad32d73-7150-40e7-a6d3-a205c81d8734 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.326068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad32d73-7150-40e7-a6d3-a205c81d8734 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.334404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f005b-b897-4c04-b509-3fac56dd94a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.335276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f005b-b897-4c04-b509-3fac56dd94a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.335557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f005b-b897-4c04-b509-3fac56dd94a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.335839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f005b-b897-4c04-b509-3fac56dd94a7 admin admin] PATH_INFO: `/v3/projects/98627493c060475fb7e8248de3c8ee8e/users/7d1033a9ccb64ac3aa30df6ba7113e41/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.363269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f005b-b897-4c04-b509-3fac56dd94a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98627493c060475fb7e8248de3c8ee8e, user_id=7d1033a9ccb64ac3aa30df6ba7113e41, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.366477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f005b-b897-4c04-b509-3fac56dd94a7 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.397099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1161/2330] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:15 2022] PUT /identity/v3/projects/98627493c060475fb7e8248de3c8ee8e/users/7d1033a9ccb64ac3aa30df6ba7113e41/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:59:15.408277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d09fda-f90e-455d-951b-294a0bab0957 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.408563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d09fda-f90e-455d-951b-294a0bab0957 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.408839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d09fda-f90e-455d-951b-294a0bab0957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.438233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e2d09fda-f90e-455d-951b-294a0bab0957 None None] MFA Rules not processed for user `7d1033a9ccb64ac3aa30df6ba7113e41`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:15.439445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e2d09fda-f90e-455d-951b-294a0bab0957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:15.517682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1170/2331] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:15.522220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f005b-b897-4c04-b509-3fac56dd94a7 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.587066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f005b-b897-4c04-b509-3fac56dd94a7 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.592372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5449939-fc06-476a-88a2-29eb1e0dd519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.592974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5449939-fc06-476a-88a2-29eb1e0dd519 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.593251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5449939-fc06-476a-88a2-29eb1e0dd519 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.593688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5449939-fc06-476a-88a2-29eb1e0dd519 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.594360 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5449939-fc06-476a-88a2-29eb1e0dd519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.596394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5449939-fc06-476a-88a2-29eb1e0dd519 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.623340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1162/2332] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:15 2022] POST /identity/v3/projects => generated 353 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:15.627379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d09fda-f90e-455d-951b-294a0bab0957 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.639422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d09fda-f90e-455d-951b-294a0bab0957 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.702236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5341eba-d1c7-4dc2-ab5c-570056177c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.702853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5341eba-d1c7-4dc2-ab5c-570056177c62 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.703136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5341eba-d1c7-4dc2-ab5c-570056177c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.703410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5341eba-d1c7-4dc2-ab5c-570056177c62 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.704048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5341eba-d1c7-4dc2-ab5c-570056177c62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.706038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5341eba-d1c7-4dc2-ab5c-570056177c62 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.730705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1171/2333] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:15 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:59:15.734763 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5449939-fc06-476a-88a2-29eb1e0dd519 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.744878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5449939-fc06-476a-88a2-29eb1e0dd519 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.749490 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb79bf-bdb7-4bc7-8a03-23439d22089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.750119 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb79bf-bdb7-4bc7-8a03-23439d22089c admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.750465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb79bf-bdb7-4bc7-8a03-23439d22089c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.750736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb79bf-bdb7-4bc7-8a03-23439d22089c admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.754300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb79bf-bdb7-4bc7-8a03-23439d22089c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.755975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb79bf-bdb7-4bc7-8a03-23439d22089c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.767579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1163/2334] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:15.771724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5341eba-d1c7-4dc2-ab5c-570056177c62 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.783142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5341eba-d1c7-4dc2-ab5c-570056177c62 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.789828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca21248-debf-4779-9f6a-f2e3b037f0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.789828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca21248-debf-4779-9f6a-f2e3b037f0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.789828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca21248-debf-4779-9f6a-f2e3b037f0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.789828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca21248-debf-4779-9f6a-f2e3b037f0a9 admin admin] PATH_INFO: `/v3/projects/1fc939e77a3b4f04a179ebe7f068820f/users/4ab76cd1ba4f4fe88cfb95ae42c460d9/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.804646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca21248-debf-4779-9f6a-f2e3b037f0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc939e77a3b4f04a179ebe7f068820f, user_id=4ab76cd1ba4f4fe88cfb95ae42c460d9, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.807338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca21248-debf-4779-9f6a-f2e3b037f0a9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.822185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1172/2335] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:15 2022] PUT /identity/v3/projects/1fc939e77a3b4f04a179ebe7f068820f/users/4ab76cd1ba4f4fe88cfb95ae42c460d9/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:15.826983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb79bf-bdb7-4bc7-8a03-23439d22089c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.837168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb79bf-bdb7-4bc7-8a03-23439d22089c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.875691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:15.897373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c712ba-ac62-4e9d-9e0d-ca562e764f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.898020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c712ba-ac62-4e9d-9e0d-ca562e764f6f admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.898293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c712ba-ac62-4e9d-9e0d-ca562e764f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.898557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c712ba-ac62-4e9d-9e0d-ca562e764f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.899219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c712ba-ac62-4e9d-9e0d-ca562e764f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.900974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c712ba-ac62-4e9d-9e0d-ca562e764f6f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.910653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1164/2336] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:59:15.914986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca21248-debf-4779-9f6a-f2e3b037f0a9 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:15.925015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca21248-debf-4779-9f6a-f2e3b037f0a9 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:15.929547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c3c01-a421-418c-b30e-6140e406104e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:15.930192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c3c01-a421-418c-b30e-6140e406104e admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.930466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c3c01-a421-418c-b30e-6140e406104e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.930738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c3c01-a421-418c-b30e-6140e406104e admin admin] PATH_INFO: `/v3/projects/1fc939e77a3b4f04a179ebe7f068820f/users/4ab76cd1ba4f4fe88cfb95ae42c460d9/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.932601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c3c01-a421-418c-b30e-6140e406104e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc939e77a3b4f04a179ebe7f068820f, user_id=4ab76cd1ba4f4fe88cfb95ae42c460d9, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:15.935822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c3c01-a421-418c-b30e-6140e406104e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:15.945824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1173/2337] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:15 2022] PUT /identity/v3/projects/1fc939e77a3b4f04a179ebe7f068820f/users/4ab76cd1ba4f4fe88cfb95ae42c460d9/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:15.957914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060fcd0-f9ea-41bc-95df-7c29370ca7b9 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:15.958509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060fcd0-f9ea-41bc-95df-7c29370ca7b9 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:15.958789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060fcd0-f9ea-41bc-95df-7c29370ca7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:15.986492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5060fcd0-f9ea-41bc-95df-7c29370ca7b9 None None] MFA Rules not processed for user `4ab76cd1ba4f4fe88cfb95ae42c460d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:15.987684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5060fcd0-f9ea-41bc-95df-7c29370ca7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:16.055616 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1165/2338] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:16.067727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c3c01-a421-418c-b30e-6140e406104e admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:16.069490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b52c3c01-a421-418c-b30e-6140e406104e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:16.110710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:16.127178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c3c01-a421-418c-b30e-6140e406104e admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:16.131276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269b788-9802-4da3-829a-bbaf33a67c83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:16.132098 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269b788-9802-4da3-829a-bbaf33a67c83 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:16.132373 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269b788-9802-4da3-829a-bbaf33a67c83 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:16.132640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269b788-9802-4da3-829a-bbaf33a67c83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:16.142441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269b788-9802-4da3-829a-bbaf33a67c83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:16.143872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269b788-9802-4da3-829a-bbaf33a67c83 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:16.153127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1174/2339] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:16 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:16.228484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060fcd0-f9ea-41bc-95df-7c29370ca7b9 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:16.238595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060fcd0-f9ea-41bc-95df-7c29370ca7b9 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:16.242155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1950033-0783-4146-bed7-416baae6bf4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:16.242752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1950033-0783-4146-bed7-416baae6bf4a service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:16.243028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1950033-0783-4146-bed7-416baae6bf4a service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:16.243297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1950033-0783-4146-bed7-416baae6bf4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:16.299697 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1950033-0783-4146-bed7-416baae6bf4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:16.301180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1950033-0783-4146-bed7-416baae6bf4a service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:16.310070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1166/2340] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:26.148972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=9 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:26.306083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=73 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:31.125031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269b788-9802-4da3-829a-bbaf33a67c83 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:31.186746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269b788-9802-4da3-829a-bbaf33a67c83 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:31.191607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:31.192237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:31.192520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:31.192795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] PATH_INFO: `/v3/users/7d1033a9ccb64ac3aa30df6ba7113e41` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:31.193957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1033a9ccb64ac3aa30df6ba7113e41)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:31.195880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:31.240919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1033a9ccb64ac3aa30df6ba7113e41'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.241615 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1033a9ccb64ac3aa30df6ba7113e41'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.248508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1033a9ccb64ac3aa30df6ba7113e41'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.260679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1033a9ccb64ac3aa30df6ba7113e41'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.267884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1033a9ccb64ac3aa30df6ba7113e41'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.274020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1175/2341] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:31 2022] DELETE /identity/v3/users/7d1033a9ccb64ac3aa30df6ba7113e41 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:31.278675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457253f-8bc2-4b54-a234-3d5c8eebfde6 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:31.279178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457253f-8bc2-4b54-a234-3d5c8eebfde6 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:31.279401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457253f-8bc2-4b54-a234-3d5c8eebfde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:31.298378 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d457253f-8bc2-4b54-a234-3d5c8eebfde6 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:31.299655 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d457253f-8bc2-4b54-a234-3d5c8eebfde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:31.358800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1167/2342] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:59:31.368334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:31.369787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:31.425012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415f7cf-6ee3-4c43-8861-a956e4234079 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:31.428562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23eacd-769b-40f5-9a80-a05227e08c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:31.429159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23eacd-769b-40f5-9a80-a05227e08c39 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:31.429437 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23eacd-769b-40f5-9a80-a05227e08c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:31.429746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23eacd-769b-40f5-9a80-a05227e08c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:31.438887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23eacd-769b-40f5-9a80-a05227e08c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:31.440305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23eacd-769b-40f5-9a80-a05227e08c39 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:31.449389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1176/2343] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:31.716554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457253f-8bc2-4b54-a234-3d5c8eebfde6 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:31.718120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d457253f-8bc2-4b54-a234-3d5c8eebfde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:31.731683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457253f-8bc2-4b54-a234-3d5c8eebfde6 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:31.738162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:31.738772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:31.739066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:31.739334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] PATH_INFO: `/v3/projects/98627493c060475fb7e8248de3c8ee8e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:31.740615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98627493c060475fb7e8248de3c8ee8e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:31.742549 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:31.788010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98627493c060475fb7e8248de3c8ee8e', 'request_id': 'req-e6e723fe-b643-4367-a6b5-5601ef6f0511'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.794964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98627493c060475fb7e8248de3c8ee8e', 'request_id': 'req-e6e723fe-b643-4367-a6b5-5601ef6f0511'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.801733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98627493c060475fb7e8248de3c8ee8e', 'request_id': 'req-e6e723fe-b643-4367-a6b5-5601ef6f0511'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.813112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] The token cache is being invalidate because project 98627493c060475fb7e8248de3c8ee8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:31.813383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.814063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.815567 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1168/2344] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:31 2022] DELETE /identity/v3/projects/98627493c060475fb7e8248de3c8ee8e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:31.819500 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23eacd-769b-40f5-9a80-a05227e08c39 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:31.829329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23eacd-769b-40f5-9a80-a05227e08c39 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:31.847363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:31.886914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:31.887540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:31.887824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:31.888100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] PATH_INFO: `/v3/users/4ab76cd1ba4f4fe88cfb95ae42c460d9` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:31.889230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab76cd1ba4f4fe88cfb95ae42c460d9)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:31.891205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:31.935662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab76cd1ba4f4fe88cfb95ae42c460d9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.936853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab76cd1ba4f4fe88cfb95ae42c460d9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.943789 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab76cd1ba4f4fe88cfb95ae42c460d9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.955366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab76cd1ba4f4fe88cfb95ae42c460d9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.962597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-39cfd19d-2996-42f0-be92-5462f10fb513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab76cd1ba4f4fe88cfb95ae42c460d9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:31.968926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1177/2345] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:31 2022] DELETE /identity/v3/users/4ab76cd1ba4f4fe88cfb95ae42c460d9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:32.239999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:32.241446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:32.254673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e723fe-b643-4367-a6b5-5601ef6f0511 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:32.315708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:32.316339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.316617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.316888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] PATH_INFO: `/v3/projects/1fc939e77a3b4f04a179ebe7f068820f` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.318019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc939e77a3b4f04a179ebe7f068820f)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:32.319923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:32.350669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:32.367896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc939e77a3b4f04a179ebe7f068820f', 'request_id': 'req-3cb16e78-983a-4328-9b3e-00a7fbae1e73'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:32.374868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc939e77a3b4f04a179ebe7f068820f', 'request_id': 'req-3cb16e78-983a-4328-9b3e-00a7fbae1e73'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:32.380982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc939e77a3b4f04a179ebe7f068820f', 'request_id': 'req-3cb16e78-983a-4328-9b3e-00a7fbae1e73'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:32.391598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] The token cache is being invalidate because project 1fc939e77a3b4f04a179ebe7f068820f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:32.391882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:32.392529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-3cb16e78-983a-4328-9b3e-00a7fbae1e73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:32.394034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1169/2346] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:32 2022] DELETE /identity/v3/projects/1fc939e77a3b4f04a179ebe7f068820f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:32.398888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffbdf2-0c25-4fc1-b461-8976f3d09b3c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.399341 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffbdf2-0c25-4fc1-b461-8976f3d09b3c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.399433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffbdf2-0c25-4fc1-b461-8976f3d09b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.419421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-c9ffbdf2-0c25-4fc1-b461-8976f3d09b3c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:32.420624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c9ffbdf2-0c25-4fc1-b461-8976f3d09b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:32.479046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1178/2347] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:32.486626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255eb3d-1590-46c3-8568-1563de77fc7e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.486909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255eb3d-1590-46c3-8568-1563de77fc7e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.487177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255eb3d-1590-46c3-8568-1563de77fc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.506165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-7255eb3d-1590-46c3-8568-1563de77fc7e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:32.507345 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7255eb3d-1590-46c3-8568-1563de77fc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:32.515595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1170/2348] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:32.519801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffbdf2-0c25-4fc1-b461-8976f3d09b3c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:32.529412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffbdf2-0c25-4fc1-b461-8976f3d09b3c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:32.534696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7813b0-49b4-4f20-9fe0-c842c27b8c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:32.535292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7813b0-49b4-4f20-9fe0-c842c27b8c64 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.535573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7813b0-49b4-4f20-9fe0-c842c27b8c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.535848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7813b0-49b4-4f20-9fe0-c842c27b8c64 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.536505 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7813b0-49b4-4f20-9fe0-c842c27b8c64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:59:32.536813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7813b0-49b4-4f20-9fe0-c842c27b8c64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:32.538467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7813b0-49b4-4f20-9fe0-c842c27b8c64 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:32.550025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1179/2349] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:59:32.554209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255eb3d-1590-46c3-8568-1563de77fc7e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:32.563290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255eb3d-1590-46c3-8568-1563de77fc7e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:32.567884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae04267-ebfb-4a39-be8c-afed6297dda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:32.568468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae04267-ebfb-4a39-be8c-afed6297dda8 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.568747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae04267-ebfb-4a39-be8c-afed6297dda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.569017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae04267-ebfb-4a39-be8c-afed6297dda8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.569679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae04267-ebfb-4a39-be8c-afed6297dda8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:32.571580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae04267-ebfb-4a39-be8c-afed6297dda8 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:32.603455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1171/2350] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:32 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:32.607541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7813b0-49b4-4f20-9fe0-c842c27b8c64 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:32.617340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7813b0-49b4-4f20-9fe0-c842c27b8c64 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:32.642707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:32.675166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2997cd5-685c-4b19-88a5-9cff878e90a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:32.675790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2997cd5-685c-4b19-88a5-9cff878e90a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.676067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2997cd5-685c-4b19-88a5-9cff878e90a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.676334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2997cd5-685c-4b19-88a5-9cff878e90a0 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.676972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2997cd5-685c-4b19-88a5-9cff878e90a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:32.678918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2997cd5-685c-4b19-88a5-9cff878e90a0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:32.699982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1180/2351] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:32 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:59:32.704759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae04267-ebfb-4a39-be8c-afed6297dda8 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:32.715043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae04267-ebfb-4a39-be8c-afed6297dda8 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:32.719638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13c32c-0eab-4353-82e9-8e5814429564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:32.720223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13c32c-0eab-4353-82e9-8e5814429564 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.720501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13c32c-0eab-4353-82e9-8e5814429564 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.720769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13c32c-0eab-4353-82e9-8e5814429564 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.721401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13c32c-0eab-4353-82e9-8e5814429564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:32.723030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13c32c-0eab-4353-82e9-8e5814429564 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:32.732672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1172/2352] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:59:32.736417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2997cd5-685c-4b19-88a5-9cff878e90a0 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:32.746198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2997cd5-685c-4b19-88a5-9cff878e90a0 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:32.751324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21af66-b135-4f05-bb24-b95e1e0db8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:32.751911 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21af66-b135-4f05-bb24-b95e1e0db8bd admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.752196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21af66-b135-4f05-bb24-b95e1e0db8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.752474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21af66-b135-4f05-bb24-b95e1e0db8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.753117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21af66-b135-4f05-bb24-b95e1e0db8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:32.754750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21af66-b135-4f05-bb24-b95e1e0db8bd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:32.765709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1181/2353] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:32.769842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13c32c-0eab-4353-82e9-8e5814429564 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:32.779447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13c32c-0eab-4353-82e9-8e5814429564 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:32.784009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b0c71-ad1e-4d5a-9638-33cefa1dfadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:32.784634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b0c71-ad1e-4d5a-9638-33cefa1dfadc admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.784910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b0c71-ad1e-4d5a-9638-33cefa1dfadc admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.785183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b0c71-ad1e-4d5a-9638-33cefa1dfadc admin admin] PATH_INFO: `/v3/projects/aef1d8cd4fac401ab6c894ae5d3b436f/users/442004b9880c497db54772c8c36abbc9/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.800216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b0c71-ad1e-4d5a-9638-33cefa1dfadc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef1d8cd4fac401ab6c894ae5d3b436f, user_id=442004b9880c497db54772c8c36abbc9, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:32.802956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b0c71-ad1e-4d5a-9638-33cefa1dfadc admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:32.816270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1173/2354] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:32 2022] PUT /identity/v3/projects/aef1d8cd4fac401ab6c894ae5d3b436f/users/442004b9880c497db54772c8c36abbc9/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:32.828245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d693748-0268-47c5-a811-dce9fa6bf52c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.828533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d693748-0268-47c5-a811-dce9fa6bf52c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.828812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d693748-0268-47c5-a811-dce9fa6bf52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.855942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2d693748-0268-47c5-a811-dce9fa6bf52c None None] MFA Rules not processed for user `442004b9880c497db54772c8c36abbc9`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:32.857147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2d693748-0268-47c5-a811-dce9fa6bf52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:32.919027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1182/2355] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:32.923592 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b0c71-ad1e-4d5a-9638-33cefa1dfadc admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:32.986477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b0c71-ad1e-4d5a-9638-33cefa1dfadc admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:32.991232 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b02db5-69c6-4de1-ba0e-3972c2dde7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:32.991835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b02db5-69c6-4de1-ba0e-3972c2dde7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:32.992113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b02db5-69c6-4de1-ba0e-3972c2dde7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:32.992382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b02db5-69c6-4de1-ba0e-3972c2dde7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:32.993009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b02db5-69c6-4de1-ba0e-3972c2dde7c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:32.994944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b02db5-69c6-4de1-ba0e-3972c2dde7c2 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:33.021617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1174/2356] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:32 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:33.025751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d693748-0268-47c5-a811-dce9fa6bf52c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:33.036448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d693748-0268-47c5-a811-dce9fa6bf52c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:33.092548 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f63b5e-4931-4d0e-96ce-098ef7ed5794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:33.093151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f63b5e-4931-4d0e-96ce-098ef7ed5794 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:33.093428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f63b5e-4931-4d0e-96ce-098ef7ed5794 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:33.093727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f63b5e-4931-4d0e-96ce-098ef7ed5794 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:33.094358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f63b5e-4931-4d0e-96ce-098ef7ed5794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:33.096253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f63b5e-4931-4d0e-96ce-098ef7ed5794 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:33.117363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1183/2357] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:33 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:59:33.121084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b02db5-69c6-4de1-ba0e-3972c2dde7c2 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:33.132773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b02db5-69c6-4de1-ba0e-3972c2dde7c2 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:33.137825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26170f-2e07-44e2-930d-b4badfaeed06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:33.138418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26170f-2e07-44e2-930d-b4badfaeed06 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:33.138701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26170f-2e07-44e2-930d-b4badfaeed06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:33.138975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26170f-2e07-44e2-930d-b4badfaeed06 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:33.139611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26170f-2e07-44e2-930d-b4badfaeed06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:33.141195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26170f-2e07-44e2-930d-b4badfaeed06 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:33.151641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1175/2358] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:33.155747 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f63b5e-4931-4d0e-96ce-098ef7ed5794 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:33.165449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f63b5e-4931-4d0e-96ce-098ef7ed5794 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:33.169981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f801576-7dc8-4b52-bac7-d3f2cb03f428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:33.171170 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f801576-7dc8-4b52-bac7-d3f2cb03f428 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:33.171445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f801576-7dc8-4b52-bac7-d3f2cb03f428 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:33.171716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f801576-7dc8-4b52-bac7-d3f2cb03f428 admin admin] PATH_INFO: `/v3/projects/acc830b0cd454c9cb721c3d4cb8af24b/users/1b4bd953a5d743eeb67d60abbc185116/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:33.185835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f801576-7dc8-4b52-bac7-d3f2cb03f428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc830b0cd454c9cb721c3d4cb8af24b, user_id=1b4bd953a5d743eeb67d60abbc185116, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:33.189008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f801576-7dc8-4b52-bac7-d3f2cb03f428 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:33.202680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1184/2359] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:33 2022] PUT /identity/v3/projects/acc830b0cd454c9cb721c3d4cb8af24b/users/1b4bd953a5d743eeb67d60abbc185116/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:33.206750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26170f-2e07-44e2-930d-b4badfaeed06 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:33.216883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26170f-2e07-44e2-930d-b4badfaeed06 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:33.237058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:33.274380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e8c47-af6a-4242-ade1-7bd2fb6dd14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:33.274974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e8c47-af6a-4242-ade1-7bd2fb6dd14b admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:33.275250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e8c47-af6a-4242-ade1-7bd2fb6dd14b admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:33.275522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e8c47-af6a-4242-ade1-7bd2fb6dd14b admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:33.276263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e8c47-af6a-4242-ade1-7bd2fb6dd14b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:33.277882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e8c47-af6a-4242-ade1-7bd2fb6dd14b admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:33.287668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1176/2360] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:33.291887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f801576-7dc8-4b52-bac7-d3f2cb03f428 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:33.298032 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:33.304579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f801576-7dc8-4b52-bac7-d3f2cb03f428 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:33.309578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c30d4-375a-44e0-a754-3ae5eb189c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:33.310234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c30d4-375a-44e0-a754-3ae5eb189c63 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:33.310516 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c30d4-375a-44e0-a754-3ae5eb189c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:33.310790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c30d4-375a-44e0-a754-3ae5eb189c63 admin admin] PATH_INFO: `/v3/projects/acc830b0cd454c9cb721c3d4cb8af24b/users/1b4bd953a5d743eeb67d60abbc185116/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:33.313240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c30d4-375a-44e0-a754-3ae5eb189c63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc830b0cd454c9cb721c3d4cb8af24b, user_id=1b4bd953a5d743eeb67d60abbc185116, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:33.315959 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c30d4-375a-44e0-a754-3ae5eb189c63 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:33.326013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1185/2361] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:33 2022] PUT /identity/v3/projects/acc830b0cd454c9cb721c3d4cb8af24b/users/1b4bd953a5d743eeb67d60abbc185116/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:33.336249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a36cd-c112-4cbd-a898-eb42aea6b529 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:33.336726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a36cd-c112-4cbd-a898-eb42aea6b529 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:33.336837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a36cd-c112-4cbd-a898-eb42aea6b529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:33.365279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-583a36cd-c112-4cbd-a898-eb42aea6b529 None None] MFA Rules not processed for user `1b4bd953a5d743eeb67d60abbc185116`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:33.366507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-583a36cd-c112-4cbd-a898-eb42aea6b529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:33.432495 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1177/2362] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:59:33.444161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c30d4-375a-44e0-a754-3ae5eb189c63 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:33.445886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-508c30d4-375a-44e0-a754-3ae5eb189c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:33.502093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c30d4-375a-44e0-a754-3ae5eb189c63 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:33.505634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e070dd1-352f-4f8e-94c6-3f9acb10afbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:33.506406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e070dd1-352f-4f8e-94c6-3f9acb10afbc service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:33.506685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e070dd1-352f-4f8e-94c6-3f9acb10afbc service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:33.506953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e070dd1-352f-4f8e-94c6-3f9acb10afbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:33.515755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e070dd1-352f-4f8e-94c6-3f9acb10afbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:33.517187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e070dd1-352f-4f8e-94c6-3f9acb10afbc service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:33.526640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1186/2363] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:33.601895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a36cd-c112-4cbd-a898-eb42aea6b529 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:33.612449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a36cd-c112-4cbd-a898-eb42aea6b529 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:33.616007 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e8b8c-d733-43d6-952a-f93b9931b3fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:33.616601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e8b8c-d733-43d6-952a-f93b9931b3fb service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:33.616876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e8b8c-d733-43d6-952a-f93b9931b3fb service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:33.617140 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e8b8c-d733-43d6-952a-f93b9931b3fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:33.673046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e8b8c-d733-43d6-952a-f93b9931b3fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:33.674503 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e8b8c-d733-43d6-952a-f93b9931b3fb service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:33.683109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1178/2364] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:36.385847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e070dd1-352f-4f8e-94c6-3f9acb10afbc service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:36.445678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e070dd1-352f-4f8e-94c6-3f9acb10afbc service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:36.450172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:36.450792 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:36.451072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:36.451343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] PATH_INFO: `/v3/users/442004b9880c497db54772c8c36abbc9` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:36.452451 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=442004b9880c497db54772c8c36abbc9)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:36.454423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:36.499438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '442004b9880c497db54772c8c36abbc9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:36.500109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '442004b9880c497db54772c8c36abbc9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:36.507164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '442004b9880c497db54772c8c36abbc9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:36.518806 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '442004b9880c497db54772c8c36abbc9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:36.525356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '442004b9880c497db54772c8c36abbc9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:36.531589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1187/2365] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:36 2022] DELETE /identity/v3/users/442004b9880c497db54772c8c36abbc9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:36.536044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee761303-2bc2-472b-9014-de8674c25fe4 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:36.536522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee761303-2bc2-472b-9014-de8674c25fe4 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:36.536624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee761303-2bc2-472b-9014-de8674c25fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:36.555412 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ee761303-2bc2-472b-9014-de8674c25fe4 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:36.556587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ee761303-2bc2-472b-9014-de8674c25fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:36.594687 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:36.613559 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1179/2366] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:36.623303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:36.624716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:36.638285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:36.684207 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a51fc3-2e74-4089-924b-38722171c607 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:36.687819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f3832-a44e-47e3-8e44-d12e3fb737cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:36.688420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f3832-a44e-47e3-8e44-d12e3fb737cb service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:36.688700 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f3832-a44e-47e3-8e44-d12e3fb737cb service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:36.688970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f3832-a44e-47e3-8e44-d12e3fb737cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:36.697941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f3832-a44e-47e3-8e44-d12e3fb737cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:36.699376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f3832-a44e-47e3-8e44-d12e3fb737cb service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:36.709109 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1188/2367] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:36.973807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee761303-2bc2-472b-9014-de8674c25fe4 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:36.975257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ee761303-2bc2-472b-9014-de8674c25fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:36.987165 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee761303-2bc2-472b-9014-de8674c25fe4 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:36.991739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:36.992372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:36.992651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:36.992922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] PATH_INFO: `/v3/projects/aef1d8cd4fac401ab6c894ae5d3b436f` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:36.994072 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aef1d8cd4fac401ab6c894ae5d3b436f)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:36.995985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:37.043839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef1d8cd4fac401ab6c894ae5d3b436f', 'request_id': 'req-482d89ac-b1d4-443e-9e4a-310a476f717a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.051146 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef1d8cd4fac401ab6c894ae5d3b436f', 'request_id': 'req-482d89ac-b1d4-443e-9e4a-310a476f717a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.057458 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef1d8cd4fac401ab6c894ae5d3b436f', 'request_id': 'req-482d89ac-b1d4-443e-9e4a-310a476f717a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.067842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] The token cache is being invalidate because project aef1d8cd4fac401ab6c894ae5d3b436f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:37.068123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.068736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.070330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1180/2368] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:36 2022] DELETE /identity/v3/projects/aef1d8cd4fac401ab6c894ae5d3b436f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:37.074455 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f3832-a44e-47e3-8e44-d12e3fb737cb service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:37.084644 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f3832-a44e-47e3-8e44-d12e3fb737cb service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:37.141938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:37.142565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:37.142855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:37.143118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] PATH_INFO: `/v3/users/1b4bd953a5d743eeb67d60abbc185116` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:37.144224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4bd953a5d743eeb67d60abbc185116)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:37.146190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:37.190997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4bd953a5d743eeb67d60abbc185116'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.191630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4bd953a5d743eeb67d60abbc185116'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.198486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4bd953a5d743eeb67d60abbc185116'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.210767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4bd953a5d743eeb67d60abbc185116'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.217879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c55abf85-0107-4db0-952a-f39f613cef7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4bd953a5d743eeb67d60abbc185116'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.224065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1189/2369] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:37 2022] DELETE /identity/v3/users/1b4bd953a5d743eeb67d60abbc185116 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:37.493788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:37.495224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:37.508883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d89ac-b1d4-443e-9e4a-310a476f717a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:37.566564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:37.567180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:37.567463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:37.567737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] PATH_INFO: `/v3/projects/acc830b0cd454c9cb721c3d4cb8af24b` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:37.568879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc830b0cd454c9cb721c3d4cb8af24b)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:37.570899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:37.619985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc830b0cd454c9cb721c3d4cb8af24b', 'request_id': 'req-ea278cb8-c82f-4b43-bd6e-a87186704fa2'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.627386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc830b0cd454c9cb721c3d4cb8af24b', 'request_id': 'req-ea278cb8-c82f-4b43-bd6e-a87186704fa2'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.633672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc830b0cd454c9cb721c3d4cb8af24b', 'request_id': 'req-ea278cb8-c82f-4b43-bd6e-a87186704fa2'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.639978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:37.644802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] The token cache is being invalidate because project acc830b0cd454c9cb721c3d4cb8af24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:37.645296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.646286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-ea278cb8-c82f-4b43-bd6e-a87186704fa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:37.647997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1181/2370] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:37 2022] DELETE /identity/v3/projects/acc830b0cd454c9cb721c3d4cb8af24b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:37.652839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0c9c9-f198-42d3-b67e-d10d79d7c4f0 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:37.653338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0c9c9-f198-42d3-b67e-d10d79d7c4f0 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:37.653442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0c9c9-f198-42d3-b67e-d10d79d7c4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:37.673097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-8ce0c9c9-f198-42d3-b67e-d10d79d7c4f0 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:37.674335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8ce0c9c9-f198-42d3-b67e-d10d79d7c4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:37.702964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:37.733848 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1190/2371] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:37.741981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5734f9-9eec-4537-99d2-36d07c3c44eb None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:37.742271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5734f9-9eec-4537-99d2-36d07c3c44eb None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:37.742548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5734f9-9eec-4537-99d2-36d07c3c44eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:37.762565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-1e5734f9-9eec-4537-99d2-36d07c3c44eb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:37.763758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1e5734f9-9eec-4537-99d2-36d07c3c44eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:37.771634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1182/2372] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:37.775701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0c9c9-f198-42d3-b67e-d10d79d7c4f0 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:37.785234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0c9c9-f198-42d3-b67e-d10d79d7c4f0 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:37.789826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d4ae5-3984-4b88-bd91-adf0b03b20b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:37.790418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4ae5-3984-4b88-bd91-adf0b03b20b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:37.790698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4ae5-3984-4b88-bd91-adf0b03b20b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:37.790969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4ae5-3984-4b88-bd91-adf0b03b20b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:37.791614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d4ae5-3984-4b88-bd91-adf0b03b20b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:59:37.792030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d4ae5-3984-4b88-bd91-adf0b03b20b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:37.793636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d4ae5-3984-4b88-bd91-adf0b03b20b2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:37.805606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1191/2373] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:59:37.809775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5734f9-9eec-4537-99d2-36d07c3c44eb None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:37.819596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5734f9-9eec-4537-99d2-36d07c3c44eb None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:37.824317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3c602-4a7f-49e9-9bb2-5a961c493255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:37.824844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3c602-4a7f-49e9-9bb2-5a961c493255 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:37.825122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3c602-4a7f-49e9-9bb2-5a961c493255 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:37.825394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3c602-4a7f-49e9-9bb2-5a961c493255 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:37.826063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3c602-4a7f-49e9-9bb2-5a961c493255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:37.827955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3c602-4a7f-49e9-9bb2-5a961c493255 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:37.854559 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1183/2374] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:37 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:37.858720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d4ae5-3984-4b88-bd91-adf0b03b20b2 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:37.868886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d4ae5-3984-4b88-bd91-adf0b03b20b2 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:37.924558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a85e4-3c3a-44d1-a8fe-08ae4f6f0c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:37.925863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a85e4-3c3a-44d1-a8fe-08ae4f6f0c07 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:37.926272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a85e4-3c3a-44d1-a8fe-08ae4f6f0c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:37.926562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a85e4-3c3a-44d1-a8fe-08ae4f6f0c07 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:37.927229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a85e4-3c3a-44d1-a8fe-08ae4f6f0c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:37.929184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a85e4-3c3a-44d1-a8fe-08ae4f6f0c07 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:37.950506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1192/2375] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:37 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:59:37.954854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3c602-4a7f-49e9-9bb2-5a961c493255 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:37.965231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3c602-4a7f-49e9-9bb2-5a961c493255 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:37.970309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a307c-deee-4a59-96df-baa38541fe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:37.970896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a307c-deee-4a59-96df-baa38541fe01 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:37.971174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a307c-deee-4a59-96df-baa38541fe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:37.971465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a307c-deee-4a59-96df-baa38541fe01 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:37.972084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a307c-deee-4a59-96df-baa38541fe01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:37.973713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a307c-deee-4a59-96df-baa38541fe01 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:37.983372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1184/2376] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:37.987802 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a85e4-3c3a-44d1-a8fe-08ae4f6f0c07 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:37.997417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a85e4-3c3a-44d1-a8fe-08ae4f6f0c07 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:38.002074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8057ac6-fc20-4e1e-982f-17512316c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:38.002661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8057ac6-fc20-4e1e-982f-17512316c962 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.002941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8057ac6-fc20-4e1e-982f-17512316c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.003214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8057ac6-fc20-4e1e-982f-17512316c962 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.003852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8057ac6-fc20-4e1e-982f-17512316c962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:38.005430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8057ac6-fc20-4e1e-982f-17512316c962 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:38.015720 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1193/2377] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:59:38.019653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a307c-deee-4a59-96df-baa38541fe01 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:38.029964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a307c-deee-4a59-96df-baa38541fe01 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:38.035084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f8ed2-997a-4264-9689-3c2c9ee63e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:38.035715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f8ed2-997a-4264-9689-3c2c9ee63e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.035994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f8ed2-997a-4264-9689-3c2c9ee63e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.036266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f8ed2-997a-4264-9689-3c2c9ee63e5f admin admin] PATH_INFO: `/v3/projects/54a016714320407688410a8d23993285/users/d12f247623794a12a89c67483d6ea1ec/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.054171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f8ed2-997a-4264-9689-3c2c9ee63e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a016714320407688410a8d23993285, user_id=d12f247623794a12a89c67483d6ea1ec, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:38.056873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f8ed2-997a-4264-9689-3c2c9ee63e5f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:38.069184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1185/2378] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:38 2022] PUT /identity/v3/projects/54a016714320407688410a8d23993285/users/d12f247623794a12a89c67483d6ea1ec/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:59:38.080411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff3320-5961-4aaa-b643-b4429f54c789 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.080638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff3320-5961-4aaa-b643-b4429f54c789 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.080915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff3320-5961-4aaa-b643-b4429f54c789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.108181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-cbff3320-5961-4aaa-b643-b4429f54c789 None None] MFA Rules not processed for user `d12f247623794a12a89c67483d6ea1ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:38.110057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-cbff3320-5961-4aaa-b643-b4429f54c789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:38.171152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1194/2379] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:38.176325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f8ed2-997a-4264-9689-3c2c9ee63e5f admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:38.238880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f8ed2-997a-4264-9689-3c2c9ee63e5f admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:38.243510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a5cb8-c7be-45f0-a078-5002c67dfe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:38.244110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a5cb8-c7be-45f0-a078-5002c67dfe16 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.244385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a5cb8-c7be-45f0-a078-5002c67dfe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.244650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a5cb8-c7be-45f0-a078-5002c67dfe16 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.245285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a5cb8-c7be-45f0-a078-5002c67dfe16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:38.247223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a5cb8-c7be-45f0-a078-5002c67dfe16 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:38.273466 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1186/2380] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:38 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:38.277605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff3320-5961-4aaa-b643-b4429f54c789 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:38.288714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff3320-5961-4aaa-b643-b4429f54c789 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:38.314405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:38.345869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d04ab-ed7d-4cc3-98bf-bc158459efef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:38.346396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d04ab-ed7d-4cc3-98bf-bc158459efef admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.346677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d04ab-ed7d-4cc3-98bf-bc158459efef admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.346950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d04ab-ed7d-4cc3-98bf-bc158459efef admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.347593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d04ab-ed7d-4cc3-98bf-bc158459efef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:38.349536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d04ab-ed7d-4cc3-98bf-bc158459efef admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:38.370046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1195/2381] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:38 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:38.374056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a5cb8-c7be-45f0-a078-5002c67dfe16 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:38.384288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a5cb8-c7be-45f0-a078-5002c67dfe16 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:38.389081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0a414-d391-4128-aad9-db87ff960b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:38.389710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0a414-d391-4128-aad9-db87ff960b49 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.389991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0a414-d391-4128-aad9-db87ff960b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.390261 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0a414-d391-4128-aad9-db87ff960b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.391457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0a414-d391-4128-aad9-db87ff960b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:38.393084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0a414-d391-4128-aad9-db87ff960b49 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:38.403532 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1187/2382] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:38.407973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d04ab-ed7d-4cc3-98bf-bc158459efef admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:38.418141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d04ab-ed7d-4cc3-98bf-bc158459efef admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:38.422708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954a107-05bb-484c-b289-68d42800bfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:38.423348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954a107-05bb-484c-b289-68d42800bfe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.423626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954a107-05bb-484c-b289-68d42800bfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.423895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954a107-05bb-484c-b289-68d42800bfe5 admin admin] PATH_INFO: `/v3/projects/4f41f6fc92d34a2b9a33bff95b80443d/users/fe4b5b296b484aaeaed18f24028c89ce/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.438250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954a107-05bb-484c-b289-68d42800bfe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f41f6fc92d34a2b9a33bff95b80443d, user_id=fe4b5b296b484aaeaed18f24028c89ce, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:38.440891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954a107-05bb-484c-b289-68d42800bfe5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:38.453830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1196/2383] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:38 2022] PUT /identity/v3/projects/4f41f6fc92d34a2b9a33bff95b80443d/users/fe4b5b296b484aaeaed18f24028c89ce/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:59:38.458109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0a414-d391-4128-aad9-db87ff960b49 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:38.467889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0a414-d391-4128-aad9-db87ff960b49 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:38.522086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c177db9-8a9d-43b0-b19b-e81ceb29137d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:38.522694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c177db9-8a9d-43b0-b19b-e81ceb29137d admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.522974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c177db9-8a9d-43b0-b19b-e81ceb29137d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.523243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c177db9-8a9d-43b0-b19b-e81ceb29137d admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.524023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c177db9-8a9d-43b0-b19b-e81ceb29137d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:38.525714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c177db9-8a9d-43b0-b19b-e81ceb29137d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:38.536548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1188/2384] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:38.540697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954a107-05bb-484c-b289-68d42800bfe5 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:38.550991 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954a107-05bb-484c-b289-68d42800bfe5 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:38.555626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790a3b2-5ac9-4cd0-89a7-5ec06130b312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:38.556274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790a3b2-5ac9-4cd0-89a7-5ec06130b312 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.556557 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790a3b2-5ac9-4cd0-89a7-5ec06130b312 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.556835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790a3b2-5ac9-4cd0-89a7-5ec06130b312 admin admin] PATH_INFO: `/v3/projects/4f41f6fc92d34a2b9a33bff95b80443d/users/fe4b5b296b484aaeaed18f24028c89ce/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.558788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790a3b2-5ac9-4cd0-89a7-5ec06130b312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f41f6fc92d34a2b9a33bff95b80443d, user_id=fe4b5b296b484aaeaed18f24028c89ce, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:38.561441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790a3b2-5ac9-4cd0-89a7-5ec06130b312 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:38.571544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1197/2385] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:38 2022] PUT /identity/v3/projects/4f41f6fc92d34a2b9a33bff95b80443d/users/fe4b5b296b484aaeaed18f24028c89ce/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:59:38.582126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eabd45-2c63-4c6e-a331-39f47309a893 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.582595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eabd45-2c63-4c6e-a331-39f47309a893 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.582959 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eabd45-2c63-4c6e-a331-39f47309a893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.590865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:38.610984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f7eabd45-2c63-4c6e-a331-39f47309a893 None None] MFA Rules not processed for user `fe4b5b296b484aaeaed18f24028c89ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:38.612116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f7eabd45-2c63-4c6e-a331-39f47309a893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:38.678112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1189/2386] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:38.688997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790a3b2-5ac9-4cd0-89a7-5ec06130b312 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:38.690710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5790a3b2-5ac9-4cd0-89a7-5ec06130b312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:38.746443 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790a3b2-5ac9-4cd0-89a7-5ec06130b312 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:38.750504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84109108-739c-4037-b23a-68eb94980772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:38.751125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84109108-739c-4037-b23a-68eb94980772 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:38.751398 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84109108-739c-4037-b23a-68eb94980772 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:38.751667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84109108-739c-4037-b23a-68eb94980772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:38.760332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84109108-739c-4037-b23a-68eb94980772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:38.761780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84109108-739c-4037-b23a-68eb94980772 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:38.771467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1198/2387] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:40.161355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eabd45-2c63-4c6e-a331-39f47309a893 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:40.224073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eabd45-2c63-4c6e-a331-39f47309a893 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:40.228444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:40.229063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:40.229340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:40.229648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] PATH_INFO: `/v3/users/d12f247623794a12a89c67483d6ea1ec` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:40.230714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d12f247623794a12a89c67483d6ea1ec)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:40.232613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:40.278300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd12f247623794a12a89c67483d6ea1ec'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.278974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd12f247623794a12a89c67483d6ea1ec'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.286063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd12f247623794a12a89c67483d6ea1ec'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.293605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd12f247623794a12a89c67483d6ea1ec'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.294295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd12f247623794a12a89c67483d6ea1ec'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.313433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1190/2388] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:40 2022] DELETE /identity/v3/users/d12f247623794a12a89c67483d6ea1ec => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:40.317835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6b108-f275-47c1-8ae1-1baddb743d42 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:40.318327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6b108-f275-47c1-8ae1-1baddb743d42 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:40.318421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6b108-f275-47c1-8ae1-1baddb743d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:40.338843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d5b6b108-f275-47c1-8ae1-1baddb743d42 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:40.340047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d5b6b108-f275-47c1-8ae1-1baddb743d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:40.396715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1199/2389] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:40.406226 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:40.407620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:40.418450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:40.468560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0c9e-ced6-40de-a75a-86a5a4aa4880 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:40.472152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de615599-82d8-456f-ad53-88ac4d0c3bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:40.472757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de615599-82d8-456f-ad53-88ac4d0c3bff service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:40.473106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de615599-82d8-456f-ad53-88ac4d0c3bff service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:40.473389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de615599-82d8-456f-ad53-88ac4d0c3bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:40.482924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de615599-82d8-456f-ad53-88ac4d0c3bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:40.484351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de615599-82d8-456f-ad53-88ac4d0c3bff service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:40.494299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1191/2390] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:40.761754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6b108-f275-47c1-8ae1-1baddb743d42 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:40.763211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d5b6b108-f275-47c1-8ae1-1baddb743d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:40.775938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6b108-f275-47c1-8ae1-1baddb743d42 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:40.780532 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:40.781262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:40.781679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:40.782113 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] PATH_INFO: `/v3/projects/54a016714320407688410a8d23993285` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:40.783392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a016714320407688410a8d23993285)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:40.785282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:40.806392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:40.832291 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a016714320407688410a8d23993285', 'request_id': 'req-f9df3535-cfe9-4bd7-bb16-519bff1527a0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.839693 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a016714320407688410a8d23993285', 'request_id': 'req-f9df3535-cfe9-4bd7-bb16-519bff1527a0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.845980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a016714320407688410a8d23993285', 'request_id': 'req-f9df3535-cfe9-4bd7-bb16-519bff1527a0'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.855923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] The token cache is being invalidate because project 54a016714320407688410a8d23993285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:40.856205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.856828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.858307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1200/2391] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:40 2022] DELETE /identity/v3/projects/54a016714320407688410a8d23993285 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:40.862734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de615599-82d8-456f-ad53-88ac4d0c3bff service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:40.872178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de615599-82d8-456f-ad53-88ac4d0c3bff service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:40.929661 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:40.930292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:40.930574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:40.930847 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] PATH_INFO: `/v3/users/fe4b5b296b484aaeaed18f24028c89ce` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:40.931964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe4b5b296b484aaeaed18f24028c89ce)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:40.933947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:40.986420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe4b5b296b484aaeaed18f24028c89ce'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.987156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe4b5b296b484aaeaed18f24028c89ce'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:40.994406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe4b5b296b484aaeaed18f24028c89ce'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:41.002554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe4b5b296b484aaeaed18f24028c89ce'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:41.003262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4fd8e8ba-b572-402e-a0b2-5c7f0238efb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe4b5b296b484aaeaed18f24028c89ce'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:41.021889 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1192/2392] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:40 2022] DELETE /identity/v3/users/fe4b5b296b484aaeaed18f24028c89ce => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:41.356514 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:41.358759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:41.371822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df3535-cfe9-4bd7-bb16-519bff1527a0 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:41.433826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:41.434392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:41.434673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:41.434946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] PATH_INFO: `/v3/projects/4f41f6fc92d34a2b9a33bff95b80443d` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:41.436171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f41f6fc92d34a2b9a33bff95b80443d)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:41.438230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:41.484736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f41f6fc92d34a2b9a33bff95b80443d', 'request_id': 'req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:41.493552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f41f6fc92d34a2b9a33bff95b80443d', 'request_id': 'req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:41.500077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f41f6fc92d34a2b9a33bff95b80443d', 'request_id': 'req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:41.510477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] The token cache is being invalidate because project 4f41f6fc92d34a2b9a33bff95b80443d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:41.510773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:41.511487 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-34fc1ac1-150d-4ad5-86ab-eefd04cb0c17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:41.513346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1201/2393] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:41 2022] DELETE /identity/v3/projects/4f41f6fc92d34a2b9a33bff95b80443d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:41.519276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0206a29-aa80-4c30-ad3d-f41b154c6856 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:41.519739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0206a29-aa80-4c30-ad3d-f41b154c6856 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:41.519838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0206a29-aa80-4c30-ad3d-f41b154c6856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:41.540815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-e0206a29-aa80-4c30-ad3d-f41b154c6856 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:41.542066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e0206a29-aa80-4c30-ad3d-f41b154c6856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:41.567953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:41.604395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1193/2394] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:59:41.613159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69e9ac-e8c8-45ff-8687-e157d6bb16f9 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:41.613534 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69e9ac-e8c8-45ff-8687-e157d6bb16f9 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:41.613815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69e9ac-e8c8-45ff-8687-e157d6bb16f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:41.633255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-0e69e9ac-e8c8-45ff-8687-e157d6bb16f9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:41.634516 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0e69e9ac-e8c8-45ff-8687-e157d6bb16f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:41.642860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1202/2395] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:41.646994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0206a29-aa80-4c30-ad3d-f41b154c6856 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:41.657232 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0206a29-aa80-4c30-ad3d-f41b154c6856 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:41.661773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ac550-664e-4e77-9e8d-ed33efb2733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:41.662360 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ac550-664e-4e77-9e8d-ed33efb2733a admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:41.662636 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ac550-664e-4e77-9e8d-ed33efb2733a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:41.662902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ac550-664e-4e77-9e8d-ed33efb2733a admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:41.663541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ac550-664e-4e77-9e8d-ed33efb2733a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:59:41.663838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ac550-664e-4e77-9e8d-ed33efb2733a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:41.665413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ac550-664e-4e77-9e8d-ed33efb2733a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:41.676739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1194/2396] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:59:41.680810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69e9ac-e8c8-45ff-8687-e157d6bb16f9 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:41.690547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69e9ac-e8c8-45ff-8687-e157d6bb16f9 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:41.695162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2f9bd-dec3-44e5-898c-97091672328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:41.695751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2f9bd-dec3-44e5-898c-97091672328f admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:41.696029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2f9bd-dec3-44e5-898c-97091672328f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:41.696300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2f9bd-dec3-44e5-898c-97091672328f admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:41.696924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2f9bd-dec3-44e5-898c-97091672328f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:41.698855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2f9bd-dec3-44e5-898c-97091672328f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:41.725863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1203/2397] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:41 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:59:41.729923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ac550-664e-4e77-9e8d-ed33efb2733a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:41.740696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ac550-664e-4e77-9e8d-ed33efb2733a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:41.798010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e7c5c-3807-4931-809e-b12c17f3d8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:41.798541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e7c5c-3807-4931-809e-b12c17f3d8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:41.798842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e7c5c-3807-4931-809e-b12c17f3d8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:41.799244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e7c5c-3807-4931-809e-b12c17f3d8b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:41.799742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e7c5c-3807-4931-809e-b12c17f3d8b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:41.801711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e7c5c-3807-4931-809e-b12c17f3d8b7 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:41.823973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1195/2398] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:41 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:59:41.827978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2f9bd-dec3-44e5-898c-97091672328f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:41.838366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2f9bd-dec3-44e5-898c-97091672328f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:41.842982 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a1d9c-7a7d-4836-8f63-e9b94d9b5b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:41.844089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a1d9c-7a7d-4836-8f63-e9b94d9b5b40 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:41.844348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a1d9c-7a7d-4836-8f63-e9b94d9b5b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:41.844617 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a1d9c-7a7d-4836-8f63-e9b94d9b5b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:41.845267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a1d9c-7a7d-4836-8f63-e9b94d9b5b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:41.846903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a1d9c-7a7d-4836-8f63-e9b94d9b5b40 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:41.849073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:41.858424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1204/2399] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:41.862935 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e7c5c-3807-4931-809e-b12c17f3d8b7 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:41.873485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e7c5c-3807-4931-809e-b12c17f3d8b7 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:41.878414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d6e9c-b638-4a91-b957-14558f9e3788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:41.879053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d6e9c-b638-4a91-b957-14558f9e3788 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:41.879353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d6e9c-b638-4a91-b957-14558f9e3788 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:41.879678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d6e9c-b638-4a91-b957-14558f9e3788 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:41.880335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d6e9c-b638-4a91-b957-14558f9e3788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:41.882281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d6e9c-b638-4a91-b957-14558f9e3788 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:41.894112 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1196/2400] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:59:41.898084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a1d9c-7a7d-4836-8f63-e9b94d9b5b40 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:41.909166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a1d9c-7a7d-4836-8f63-e9b94d9b5b40 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:41.913794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aa9dc-5eac-40b6-94f7-0272a340de24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:41.914445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aa9dc-5eac-40b6-94f7-0272a340de24 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:41.914750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aa9dc-5eac-40b6-94f7-0272a340de24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:41.915023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aa9dc-5eac-40b6-94f7-0272a340de24 admin admin] PATH_INFO: `/v3/projects/a97be3872f8441e59b83ad8b6e50edda/users/0a6afceacb3548ab9803fc123f859a67/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:41.930704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aa9dc-5eac-40b6-94f7-0272a340de24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a97be3872f8441e59b83ad8b6e50edda, user_id=0a6afceacb3548ab9803fc123f859a67, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:41.933379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aa9dc-5eac-40b6-94f7-0272a340de24 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:41.947836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1205/2401] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:41 2022] PUT /identity/v3/projects/a97be3872f8441e59b83ad8b6e50edda/users/0a6afceacb3548ab9803fc123f859a67/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:41.959797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267676e6-89eb-4dcb-b154-f59f5986dd42 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:41.960276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267676e6-89eb-4dcb-b154-f59f5986dd42 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:41.960621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267676e6-89eb-4dcb-b154-f59f5986dd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:41.992540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-267676e6-89eb-4dcb-b154-f59f5986dd42 None None] MFA Rules not processed for user `0a6afceacb3548ab9803fc123f859a67`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:41.993893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-267676e6-89eb-4dcb-b154-f59f5986dd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:42.068166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1197/2402] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:42.072670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aa9dc-5eac-40b6-94f7-0272a340de24 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:42.136209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aa9dc-5eac-40b6-94f7-0272a340de24 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:42.141444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd8456-7da3-460f-b11c-99c61f6d5bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:42.142079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd8456-7da3-460f-b11c-99c61f6d5bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:42.142363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd8456-7da3-460f-b11c-99c61f6d5bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:42.142656 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd8456-7da3-460f-b11c-99c61f6d5bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:42.143280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd8456-7da3-460f-b11c-99c61f6d5bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:42.145194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd8456-7da3-460f-b11c-99c61f6d5bb1 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:42.175382 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1206/2403] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:42 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:59:42.179255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267676e6-89eb-4dcb-b154-f59f5986dd42 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:42.190451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267676e6-89eb-4dcb-b154-f59f5986dd42 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:42.217879 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:42.268561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a5da6-4c25-452a-801c-664f3b6355e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:42.269179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a5da6-4c25-452a-801c-664f3b6355e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:42.269461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a5da6-4c25-452a-801c-664f3b6355e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:42.269764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a5da6-4c25-452a-801c-664f3b6355e8 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:42.270423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a5da6-4c25-452a-801c-664f3b6355e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:42.272351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a5da6-4c25-452a-801c-664f3b6355e8 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:42.298554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1198/2404] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:42 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 14:59:42.299752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd8456-7da3-460f-b11c-99c61f6d5bb1 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:42.310672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd8456-7da3-460f-b11c-99c61f6d5bb1 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:42.317374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94ec76-bfaa-4e67-9bb9-c94a9a1e9828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:42.317678 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94ec76-bfaa-4e67-9bb9-c94a9a1e9828 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:42.317891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94ec76-bfaa-4e67-9bb9-c94a9a1e9828 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:42.318132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94ec76-bfaa-4e67-9bb9-c94a9a1e9828 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:42.318794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94ec76-bfaa-4e67-9bb9-c94a9a1e9828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:42.320404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94ec76-bfaa-4e67-9bb9-c94a9a1e9828 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:42.335093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1207/2405] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:42.338365 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a5da6-4c25-452a-801c-664f3b6355e8 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:42.350371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a5da6-4c25-452a-801c-664f3b6355e8 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:42.355171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32414143-c293-4a0e-8aab-34bcf3d94a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:42.356051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32414143-c293-4a0e-8aab-34bcf3d94a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:42.356387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32414143-c293-4a0e-8aab-34bcf3d94a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:42.356719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32414143-c293-4a0e-8aab-34bcf3d94a7d admin admin] PATH_INFO: `/v3/projects/c4ac92308d0e4e6fb75c8d042a45bb28/users/23801c9635164e9abfea1dfba2bf1e16/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:42.372273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32414143-c293-4a0e-8aab-34bcf3d94a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ac92308d0e4e6fb75c8d042a45bb28, user_id=23801c9635164e9abfea1dfba2bf1e16, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:42.375107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32414143-c293-4a0e-8aab-34bcf3d94a7d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:42.388734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1199/2406] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:42 2022] PUT /identity/v3/projects/c4ac92308d0e4e6fb75c8d042a45bb28/users/23801c9635164e9abfea1dfba2bf1e16/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:42.393234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94ec76-bfaa-4e67-9bb9-c94a9a1e9828 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:42.403336 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94ec76-bfaa-4e67-9bb9-c94a9a1e9828 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:42.464106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061f355-ed3d-4ecb-99a4-de36a55c4dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:42.464810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f355-ed3d-4ecb-99a4-de36a55c4dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:42.465164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f355-ed3d-4ecb-99a4-de36a55c4dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:42.465530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f355-ed3d-4ecb-99a4-de36a55c4dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:42.466386 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061f355-ed3d-4ecb-99a4-de36a55c4dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:42.468790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061f355-ed3d-4ecb-99a4-de36a55c4dd0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:42.482329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1208/2407] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:42.484288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32414143-c293-4a0e-8aab-34bcf3d94a7d admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:42.495949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32414143-c293-4a0e-8aab-34bcf3d94a7d admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:42.502601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc8086-700c-4533-a4c8-464b3067a685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:42.503270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc8086-700c-4533-a4c8-464b3067a685 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:42.503554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc8086-700c-4533-a4c8-464b3067a685 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:42.503824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc8086-700c-4533-a4c8-464b3067a685 admin admin] PATH_INFO: `/v3/projects/c4ac92308d0e4e6fb75c8d042a45bb28/users/23801c9635164e9abfea1dfba2bf1e16/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:42.505810 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc8086-700c-4533-a4c8-464b3067a685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ac92308d0e4e6fb75c8d042a45bb28, user_id=23801c9635164e9abfea1dfba2bf1e16, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:42.508452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc8086-700c-4533-a4c8-464b3067a685 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:42.523840 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1200/2408] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:42 2022] PUT /identity/v3/projects/c4ac92308d0e4e6fb75c8d042a45bb28/users/23801c9635164e9abfea1dfba2bf1e16/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:42.534631 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc33d0-f1e3-42d9-b6ac-b6bf7250ed49 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:42.534923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc33d0-f1e3-42d9-b6ac-b6bf7250ed49 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:42.535200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc33d0-f1e3-42d9-b6ac-b6bf7250ed49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:42.565474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-0edc33d0-f1e3-42d9-b6ac-b6bf7250ed49 None None] MFA Rules not processed for user `23801c9635164e9abfea1dfba2bf1e16`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:42.571411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0edc33d0-f1e3-42d9-b6ac-b6bf7250ed49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:42.614112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:42.667529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1209/2409] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:59:42.678677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc8086-700c-4533-a4c8-464b3067a685 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:42.680665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-dfdc8086-700c-4533-a4c8-464b3067a685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:42.759972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc8086-700c-4533-a4c8-464b3067a685 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:42.763927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a1902-6502-48e8-808a-fff5e98cfa9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:42.764676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a1902-6502-48e8-808a-fff5e98cfa9a service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:42.765695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a1902-6502-48e8-808a-fff5e98cfa9a service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:42.766074 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a1902-6502-48e8-808a-fff5e98cfa9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:42.845933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a1902-6502-48e8-808a-fff5e98cfa9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:42.848520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a1902-6502-48e8-808a-fff5e98cfa9a service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:42.862947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1201/2410] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:45.738558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b55d6e-aa68-4c8a-b521-458964c9a792 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:45.739047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b55d6e-aa68-4c8a-b521-458964c9a792 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:45.739165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b55d6e-aa68-4c8a-b521-458964c9a792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:45.758570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b9b55d6e-aa68-4c8a-b521-458964c9a792 None None] MFA Rules not processed for user `0be78e1abb3947a2ac5881b9fd2f6fae`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:45.759825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b9b55d6e-aa68-4c8a-b521-458964c9a792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:45.815856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1210/2411] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:47.178565 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a1902-6502-48e8-808a-fff5e98cfa9a service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:47.187058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a1902-6502-48e8-808a-fff5e98cfa9a service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:47.191277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dada535-b636-4e73-9e6e-256445c5ca05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:47.191875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dada535-b636-4e73-9e6e-256445c5ca05 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:47.192154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dada535-b636-4e73-9e6e-256445c5ca05 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:47.192423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dada535-b636-4e73-9e6e-256445c5ca05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:47.201232 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dada535-b636-4e73-9e6e-256445c5ca05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:47.202707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dada535-b636-4e73-9e6e-256445c5ca05 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:47.213030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1202/2412] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:52.165530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b55d6e-aa68-4c8a-b521-458964c9a792 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:52.227524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b55d6e-aa68-4c8a-b521-458964c9a792 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:52.231927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:52.232554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:52.232834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:52.233108 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] PATH_INFO: `/v3/users/0a6afceacb3548ab9803fc123f859a67` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:52.234248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a6afceacb3548ab9803fc123f859a67)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:52.236698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:52.285226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a6afceacb3548ab9803fc123f859a67'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:52.285890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a6afceacb3548ab9803fc123f859a67'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:52.292905 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a6afceacb3548ab9803fc123f859a67'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:52.306211 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a6afceacb3548ab9803fc123f859a67'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:52.308658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:52.314155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a6afceacb3548ab9803fc123f859a67'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:52.320747 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1211/2413] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:52 2022] DELETE /identity/v3/users/0a6afceacb3548ab9803fc123f859a67 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:52.325306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75359171-4c32-4f73-bdea-6915d2c4d0cd None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:52.325791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75359171-4c32-4f73-bdea-6915d2c4d0cd None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:52.325887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75359171-4c32-4f73-bdea-6915d2c4d0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:52.345641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-75359171-4c32-4f73-bdea-6915d2c4d0cd None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:52.347020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-75359171-4c32-4f73-bdea-6915d2c4d0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:52.359101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:52.405011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1203/2414] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:52.470027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:52.471591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:52.528133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de71f-c5fd-4fec-9d04-466e52a616ea admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:52.532366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04686700-35a4-430a-9ae8-2437654dbf57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:52.532972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04686700-35a4-430a-9ae8-2437654dbf57 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:52.533254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04686700-35a4-430a-9ae8-2437654dbf57 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:52.533549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04686700-35a4-430a-9ae8-2437654dbf57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:52.542429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04686700-35a4-430a-9ae8-2437654dbf57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:52.543852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04686700-35a4-430a-9ae8-2437654dbf57 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:52.553123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1212/2415] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:52.926730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75359171-4c32-4f73-bdea-6915d2c4d0cd None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:52.928746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-75359171-4c32-4f73-bdea-6915d2c4d0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:53.004250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75359171-4c32-4f73-bdea-6915d2c4d0cd None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:53.008863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:53.009484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:53.009786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:53.010058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] PATH_INFO: `/v3/projects/a97be3872f8441e59b83ad8b6e50edda` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:53.011162 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a97be3872f8441e59b83ad8b6e50edda)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:53.013059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:53.059185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a97be3872f8441e59b83ad8b6e50edda', 'request_id': 'req-e61d362f-e15d-4c4e-9d9d-25ef78724598'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.066478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a97be3872f8441e59b83ad8b6e50edda', 'request_id': 'req-e61d362f-e15d-4c4e-9d9d-25ef78724598'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.073017 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a97be3872f8441e59b83ad8b6e50edda', 'request_id': 'req-e61d362f-e15d-4c4e-9d9d-25ef78724598'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.084811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] The token cache is being invalidate because project a97be3872f8441e59b83ad8b6e50edda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:53.085099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.085803 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.087326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1204/2416] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:52 2022] DELETE /identity/v3/projects/a97be3872f8441e59b83ad8b6e50edda => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:59:53.091330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04686700-35a4-430a-9ae8-2437654dbf57 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:53.133747 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04686700-35a4-430a-9ae8-2437654dbf57 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:53.189446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:53.190123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:53.190367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:53.190640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] PATH_INFO: `/v3/users/23801c9635164e9abfea1dfba2bf1e16` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:53.191761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23801c9635164e9abfea1dfba2bf1e16)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:53.193756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:53.239929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23801c9635164e9abfea1dfba2bf1e16'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.240576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23801c9635164e9abfea1dfba2bf1e16'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.247648 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23801c9635164e9abfea1dfba2bf1e16'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.255607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:DELETE=9,keystone:SELECT=91 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:53.261547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23801c9635164e9abfea1dfba2bf1e16'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.269342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-820df842-b804-487b-a9db-b09ac62c489a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23801c9635164e9abfea1dfba2bf1e16'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.275815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1213/2417] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:53 2022] DELETE /identity/v3/users/23801c9635164e9abfea1dfba2bf1e16 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:53.556918 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:53.558385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:53.573024 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d362f-e15d-4c4e-9d9d-25ef78724598 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:53.613479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:53.630746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:53.631372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:53.631643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:53.631911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] PATH_INFO: `/v3/projects/c4ac92308d0e4e6fb75c8d042a45bb28` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:53.633010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4ac92308d0e4e6fb75c8d042a45bb28)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:53.634928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:53.680038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4ac92308d0e4e6fb75c8d042a45bb28', 'request_id': 'req-937d88f3-87b1-463e-abea-f4e04b89c79f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.687899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4ac92308d0e4e6fb75c8d042a45bb28', 'request_id': 'req-937d88f3-87b1-463e-abea-f4e04b89c79f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.694830 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4ac92308d0e4e6fb75c8d042a45bb28', 'request_id': 'req-937d88f3-87b1-463e-abea-f4e04b89c79f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.705116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] The token cache is being invalidate because project c4ac92308d0e4e6fb75c8d042a45bb28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:53.705399 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.706051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-937d88f3-87b1-463e-abea-f4e04b89c79f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:53.707767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1205/2418] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:53 2022] DELETE /identity/v3/projects/c4ac92308d0e4e6fb75c8d042a45bb28 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:53.713635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e9ac3-126e-47eb-b4da-cda88ef06543 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:53.713932 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e9ac3-126e-47eb-b4da-cda88ef06543 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:53.714212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e9ac3-126e-47eb-b4da-cda88ef06543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:53.771378 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-935e9ac3-126e-47eb-b4da-cda88ef06543 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:53.772591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-935e9ac3-126e-47eb-b4da-cda88ef06543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:53.830233 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1214/2419] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:53.838693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78070036-8094-419e-9636-1e6fc4a4cfa4 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:53.838979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78070036-8094-419e-9636-1e6fc4a4cfa4 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:53.839303 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78070036-8094-419e-9636-1e6fc4a4cfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:53.858950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-78070036-8094-419e-9636-1e6fc4a4cfa4 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:53.860896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-78070036-8094-419e-9636-1e6fc4a4cfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:53.868781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1206/2420] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:53.873005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e9ac3-126e-47eb-b4da-cda88ef06543 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:53.882756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e9ac3-126e-47eb-b4da-cda88ef06543 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:53.887419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f4c16-ae1b-430a-a8e7-836e33b66b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:53.888017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f4c16-ae1b-430a-a8e7-836e33b66b57 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:53.888323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f4c16-ae1b-430a-a8e7-836e33b66b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:53.888594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f4c16-ae1b-430a-a8e7-836e33b66b57 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:53.889242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f4c16-ae1b-430a-a8e7-836e33b66b57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:59:53.889576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f4c16-ae1b-430a-a8e7-836e33b66b57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:53.891169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f4c16-ae1b-430a-a8e7-836e33b66b57 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:53.902619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1215/2421] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:59:53.907507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78070036-8094-419e-9636-1e6fc4a4cfa4 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:53.916488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78070036-8094-419e-9636-1e6fc4a4cfa4 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:53.921082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400df06-c114-4c7d-bfe1-fb6af08e162d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:53.921725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400df06-c114-4c7d-bfe1-fb6af08e162d admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:53.922004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400df06-c114-4c7d-bfe1-fb6af08e162d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:53.922275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400df06-c114-4c7d-bfe1-fb6af08e162d admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:53.923547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400df06-c114-4c7d-bfe1-fb6af08e162d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:53.925438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400df06-c114-4c7d-bfe1-fb6af08e162d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:53.956772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1207/2422] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:53 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:53.958283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f4c16-ae1b-430a-a8e7-836e33b66b57 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:53.968335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f4c16-ae1b-430a-a8e7-836e33b66b57 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.028111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9256f-d4cf-44d1-886f-d99d1d3216e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.028706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9256f-d4cf-44d1-886f-d99d1d3216e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.028994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9256f-d4cf-44d1-886f-d99d1d3216e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.029278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9256f-d4cf-44d1-886f-d99d1d3216e6 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.029941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9256f-d4cf-44d1-886f-d99d1d3216e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.031960 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9256f-d4cf-44d1-886f-d99d1d3216e6 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.054476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1216/2423] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:53 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:54.058561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400df06-c114-4c7d-bfe1-fb6af08e162d admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:54.069151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400df06-c114-4c7d-bfe1-fb6af08e162d admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.073942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d13794-42bd-4ded-b62d-f9988310bcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.074564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d13794-42bd-4ded-b62d-f9988310bcfb admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.074832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d13794-42bd-4ded-b62d-f9988310bcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.075108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d13794-42bd-4ded-b62d-f9988310bcfb admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.075812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d13794-42bd-4ded-b62d-f9988310bcfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.077419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d13794-42bd-4ded-b62d-f9988310bcfb admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.088293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1208/2424] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:54.092254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9256f-d4cf-44d1-886f-d99d1d3216e6 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:54.103006 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9256f-d4cf-44d1-886f-d99d1d3216e6 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.107591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575f798-224c-4bd8-bf61-232a448095eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.108185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575f798-224c-4bd8-bf61-232a448095eb admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.108464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575f798-224c-4bd8-bf61-232a448095eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.108738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575f798-224c-4bd8-bf61-232a448095eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.109379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575f798-224c-4bd8-bf61-232a448095eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.111014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575f798-224c-4bd8-bf61-232a448095eb admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.121426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1217/2425] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:54.125518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d13794-42bd-4ded-b62d-f9988310bcfb admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:54.135447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d13794-42bd-4ded-b62d-f9988310bcfb admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.140253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0537fe-7abc-4727-b66c-5ba9e9dbc3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.140998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0537fe-7abc-4727-b66c-5ba9e9dbc3d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.141208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0537fe-7abc-4727-b66c-5ba9e9dbc3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.142055 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0537fe-7abc-4727-b66c-5ba9e9dbc3d1 admin admin] PATH_INFO: `/v3/projects/6a1313b539b548e293810483fa9597b5/users/4f7c69284d004c2ca169c167088c3620/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.156211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0537fe-7abc-4727-b66c-5ba9e9dbc3d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1313b539b548e293810483fa9597b5, user_id=4f7c69284d004c2ca169c167088c3620, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.159509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0537fe-7abc-4727-b66c-5ba9e9dbc3d1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.171421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1209/2426] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:54 2022] PUT /identity/v3/projects/6a1313b539b548e293810483fa9597b5/users/4f7c69284d004c2ca169c167088c3620/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:59:54.182277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0af612-6cdc-40ca-a1f4-a3851959d8a5 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.182567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0af612-6cdc-40ca-a1f4-a3851959d8a5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.182844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0af612-6cdc-40ca-a1f4-a3851959d8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.210513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ba0af612-6cdc-40ca-a1f4-a3851959d8a5 None None] MFA Rules not processed for user `4f7c69284d004c2ca169c167088c3620`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:54.211721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ba0af612-6cdc-40ca-a1f4-a3851959d8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:54.216724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:54.275199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1218/2427] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:54.279824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0537fe-7abc-4727-b66c-5ba9e9dbc3d1 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:54.352695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0537fe-7abc-4727-b66c-5ba9e9dbc3d1 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.357370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe59856-38e5-4a8b-b1f6-db468a855398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.358480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe59856-38e5-4a8b-b1f6-db468a855398 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.358754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe59856-38e5-4a8b-b1f6-db468a855398 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.359027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe59856-38e5-4a8b-b1f6-db468a855398 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.359675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe59856-38e5-4a8b-b1f6-db468a855398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.361665 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe59856-38e5-4a8b-b1f6-db468a855398 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.376421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:54.393430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1210/2428] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:54 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:54.402798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0af612-6cdc-40ca-a1f4-a3851959d8a5 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:54.414074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0af612-6cdc-40ca-a1f4-a3851959d8a5 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.471826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f781eb-2fd3-4a53-9d83-ba54f1b631f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.472445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f781eb-2fd3-4a53-9d83-ba54f1b631f0 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.473378 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f781eb-2fd3-4a53-9d83-ba54f1b631f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.473378 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f781eb-2fd3-4a53-9d83-ba54f1b631f0 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.473910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f781eb-2fd3-4a53-9d83-ba54f1b631f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.475584 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f781eb-2fd3-4a53-9d83-ba54f1b631f0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.497968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1219/2429] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:54 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:54.502288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe59856-38e5-4a8b-b1f6-db468a855398 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:54.512554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe59856-38e5-4a8b-b1f6-db468a855398 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.518209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd22eaa-7e47-4aee-972e-d40de5725da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.518814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd22eaa-7e47-4aee-972e-d40de5725da6 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.519087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd22eaa-7e47-4aee-972e-d40de5725da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.519350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd22eaa-7e47-4aee-972e-d40de5725da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.519989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd22eaa-7e47-4aee-972e-d40de5725da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.521742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd22eaa-7e47-4aee-972e-d40de5725da6 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.534234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1211/2430] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:54.538599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f781eb-2fd3-4a53-9d83-ba54f1b631f0 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:54.549095 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f781eb-2fd3-4a53-9d83-ba54f1b631f0 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.553812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31139336-df55-455d-9e83-5c74209299be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.554457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31139336-df55-455d-9e83-5c74209299be admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.554741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31139336-df55-455d-9e83-5c74209299be admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.555034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31139336-df55-455d-9e83-5c74209299be admin admin] PATH_INFO: `/v3/projects/ee1177626cab4115ad7b2cdc3d94bf5e/users/d6e400aeef4d4dfda6bb73991b82d814/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.570087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31139336-df55-455d-9e83-5c74209299be admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1177626cab4115ad7b2cdc3d94bf5e, user_id=d6e400aeef4d4dfda6bb73991b82d814, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.572740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31139336-df55-455d-9e83-5c74209299be admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.586133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1220/2431] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:54 2022] PUT /identity/v3/projects/ee1177626cab4115ad7b2cdc3d94bf5e/users/d6e400aeef4d4dfda6bb73991b82d814/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:54.590003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd22eaa-7e47-4aee-972e-d40de5725da6 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:54.600065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd22eaa-7e47-4aee-972e-d40de5725da6 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.660793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b0ec2-df55-46dd-9233-c735ee9f50bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.661449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b0ec2-df55-46dd-9233-c735ee9f50bc admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.661757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b0ec2-df55-46dd-9233-c735ee9f50bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.662027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b0ec2-df55-46dd-9233-c735ee9f50bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.662662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b0ec2-df55-46dd-9233-c735ee9f50bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.664290 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b0ec2-df55-46dd-9233-c735ee9f50bc admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.673776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1212/2432] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:54.677980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31139336-df55-455d-9e83-5c74209299be admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:54.688587 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31139336-df55-455d-9e83-5c74209299be admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.693316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4c8f1-9297-478f-851d-9eb2bd7f0e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.693995 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4c8f1-9297-478f-851d-9eb2bd7f0e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.694273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4c8f1-9297-478f-851d-9eb2bd7f0e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.694543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4c8f1-9297-478f-851d-9eb2bd7f0e1c admin admin] PATH_INFO: `/v3/projects/ee1177626cab4115ad7b2cdc3d94bf5e/users/d6e400aeef4d4dfda6bb73991b82d814/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.696534 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4c8f1-9297-478f-851d-9eb2bd7f0e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1177626cab4115ad7b2cdc3d94bf5e, user_id=d6e400aeef4d4dfda6bb73991b82d814, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.699192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4c8f1-9297-478f-851d-9eb2bd7f0e1c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.709993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1221/2433] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:54 2022] PUT /identity/v3/projects/ee1177626cab4115ad7b2cdc3d94bf5e/users/d6e400aeef4d4dfda6bb73991b82d814/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:54.721120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701caed-bcdd-4f8f-b37b-a8b316d600ff None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.721489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701caed-bcdd-4f8f-b37b-a8b316d600ff None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.721849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701caed-bcdd-4f8f-b37b-a8b316d600ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.749530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d701caed-bcdd-4f8f-b37b-a8b316d600ff None None] MFA Rules not processed for user `d6e400aeef4d4dfda6bb73991b82d814`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:54.751445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d701caed-bcdd-4f8f-b37b-a8b316d600ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:54.819965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1213/2434] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:54.831148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4c8f1-9297-478f-851d-9eb2bd7f0e1c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:54.833105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8ec4c8f1-9297-478f-851d-9eb2bd7f0e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:54.894506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:54.905844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4c8f1-9297-478f-851d-9eb2bd7f0e1c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:54.909475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc82ca-c96e-4464-8b48-129e1c940162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:54.910100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc82ca-c96e-4464-8b48-129e1c940162 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:54.910376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc82ca-c96e-4464-8b48-129e1c940162 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:54.910645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc82ca-c96e-4464-8b48-129e1c940162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:54.964542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc82ca-c96e-4464-8b48-129e1c940162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:54.965931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc82ca-c96e-4464-8b48-129e1c940162 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:54.974610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1222/2435] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:57.575298 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701caed-bcdd-4f8f-b37b-a8b316d600ff None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:57.682574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701caed-bcdd-4f8f-b37b-a8b316d600ff None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:57.687389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:57.687894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:57.688693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:57.689016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] PATH_INFO: `/v3/users/4f7c69284d004c2ca169c167088c3620` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:57.691509 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7c69284d004c2ca169c167088c3620)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:57.693518 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:57.701409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:57.778999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7c69284d004c2ca169c167088c3620'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:57.779580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7c69284d004c2ca169c167088c3620'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:57.786967 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7c69284d004c2ca169c167088c3620'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:57.796841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7c69284d004c2ca169c167088c3620'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:57.802747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7c69284d004c2ca169c167088c3620'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:57.822073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1214/2436] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:57 2022] DELETE /identity/v3/users/4f7c69284d004c2ca169c167088c3620 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:57.826509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86274484-b58a-4b56-948b-ea57fc14e135 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:57.826979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86274484-b58a-4b56-948b-ea57fc14e135 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:57.827163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86274484-b58a-4b56-948b-ea57fc14e135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:57.847512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-86274484-b58a-4b56-948b-ea57fc14e135 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:57.848796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-86274484-b58a-4b56-948b-ea57fc14e135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:57.920861 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1223/2437] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:59:57.927291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:57.928689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:58.008522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0915a2a-5267-49df-8eb3-bc2ad1c0bbf0 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:58.012706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a622342-945c-413c-804f-b47f9ebd6b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:58.012706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a622342-945c-413c-804f-b47f9ebd6b3e service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:58.012947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a622342-945c-413c-804f-b47f9ebd6b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:58.013093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a622342-945c-413c-804f-b47f9ebd6b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:58.022741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a622342-945c-413c-804f-b47f9ebd6b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:58.024200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a622342-945c-413c-804f-b47f9ebd6b3e service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:58.034360 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1215/2438] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 14:59:58.347772 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86274484-b58a-4b56-948b-ea57fc14e135 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:58.349310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-86274484-b58a-4b56-948b-ea57fc14e135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:58.366256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86274484-b58a-4b56-948b-ea57fc14e135 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:58.374530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:58.375295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:58.375698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:58.375886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] PATH_INFO: `/v3/projects/6a1313b539b548e293810483fa9597b5` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:58.377088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a1313b539b548e293810483fa9597b5)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:58.379134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:58.440374 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1313b539b548e293810483fa9597b5', 'request_id': 'req-58a6fde4-a6b4-46e2-a3be-37286b1de11a'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:58.449231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1313b539b548e293810483fa9597b5', 'request_id': 'req-58a6fde4-a6b4-46e2-a3be-37286b1de11a'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:58.457805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1313b539b548e293810483fa9597b5', 'request_id': 'req-58a6fde4-a6b4-46e2-a3be-37286b1de11a'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:58.469031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] The token cache is being invalidate because project 6a1313b539b548e293810483fa9597b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:58.469897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:58.471658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:58.472288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1224/2439] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:58 2022] DELETE /identity/v3/projects/6a1313b539b548e293810483fa9597b5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:58.476221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a622342-945c-413c-804f-b47f9ebd6b3e service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:58.486662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a622342-945c-413c-804f-b47f9ebd6b3e service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:58.551505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:58.552287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:58.552427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:58.552675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] PATH_INFO: `/v3/users/d6e400aeef4d4dfda6bb73991b82d814` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:58.554022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e400aeef4d4dfda6bb73991b82d814)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:58.556726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:58.594064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:58.621911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e400aeef4d4dfda6bb73991b82d814'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:58.622768 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e400aeef4d4dfda6bb73991b82d814'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:58.630765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e400aeef4d4dfda6bb73991b82d814'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:58.641867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e400aeef4d4dfda6bb73991b82d814'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:58.642491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-91fb68b1-4fe3-4a2a-ad29-68e0ed0d8dd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e400aeef4d4dfda6bb73991b82d814'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:58.661547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1216/2440] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 14:59:58 2022] DELETE /identity/v3/users/d6e400aeef4d4dfda6bb73991b82d814 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 14:59:58.998436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:58.999922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:59.013444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6fde4-a6b4-46e2-a3be-37286b1de11a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:59.041869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:59.087212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:59.087773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.088059 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.088622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] PATH_INFO: `/v3/projects/ee1177626cab4115ad7b2cdc3d94bf5e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.089841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1177626cab4115ad7b2cdc3d94bf5e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:59.091798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:59.146971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1177626cab4115ad7b2cdc3d94bf5e', 'request_id': 'req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:59.156223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1177626cab4115ad7b2cdc3d94bf5e', 'request_id': 'req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:59.164509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1177626cab4115ad7b2cdc3d94bf5e', 'request_id': 'req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:59.175189 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] The token cache is being invalidate because project ee1177626cab4115ad7b2cdc3d94bf5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 14:59:59.175480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:59.176133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5ffb4dde-7eaa-4c7a-aff0-7c5d8b12cdff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 14:59:59.177669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1225/2441] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 14:59:58 2022] DELETE /identity/v3/projects/ee1177626cab4115ad7b2cdc3d94bf5e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:59:59.182750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1497a-19ae-4a2e-9ee4-ffb0a0c642a5 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.183033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1497a-19ae-4a2e-9ee4-ffb0a0c642a5 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.183302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1497a-19ae-4a2e-9ee4-ffb0a0c642a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.204222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b8d1497a-19ae-4a2e-9ee4-ffb0a0c642a5 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:59.205438 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b8d1497a-19ae-4a2e-9ee4-ffb0a0c642a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:59.268296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1217/2442] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:59:59.275869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa0f82-6605-4ee3-839d-b01a538fd3ad None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.276239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa0f82-6605-4ee3-839d-b01a538fd3ad None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.276703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa0f82-6605-4ee3-839d-b01a538fd3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.301280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-5faa0f82-6605-4ee3-839d-b01a538fd3ad None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:59.302722 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5faa0f82-6605-4ee3-839d-b01a538fd3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:59.310784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1226/2443] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 14:59:59.315636 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1497a-19ae-4a2e-9ee4-ffb0a0c642a5 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:59.325699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1497a-19ae-4a2e-9ee4-ffb0a0c642a5 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:59.330322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9dbbd-e8d0-4a82-8615-0fd9d99555db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:59.330929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9dbbd-e8d0-4a82-8615-0fd9d99555db admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.331210 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9dbbd-e8d0-4a82-8615-0fd9d99555db admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.331477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9dbbd-e8d0-4a82-8615-0fd9d99555db admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.332132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9dbbd-e8d0-4a82-8615-0fd9d99555db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 14:59:59.332690 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9dbbd-e8d0-4a82-8615-0fd9d99555db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:59.334453 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9dbbd-e8d0-4a82-8615-0fd9d99555db admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:59.347291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1218/2444] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 14:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 14:59:59.361347 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa0f82-6605-4ee3-839d-b01a538fd3ad None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:59.370569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa0f82-6605-4ee3-839d-b01a538fd3ad None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:59.376421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282eefe-6275-4a31-b270-8bdbcf05845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:59.377038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eefe-6275-4a31-b270-8bdbcf05845f admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.377300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eefe-6275-4a31-b270-8bdbcf05845f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.377654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eefe-6275-4a31-b270-8bdbcf05845f admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.378428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282eefe-6275-4a31-b270-8bdbcf05845f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:59.380420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282eefe-6275-4a31-b270-8bdbcf05845f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:59.408081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1227/2445] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 14:59:59 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:59.412333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9dbbd-e8d0-4a82-8615-0fd9d99555db admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:59.423087 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9dbbd-e8d0-4a82-8615-0fd9d99555db admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:59.489605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ec03-2db6-49e2-8db1-a19ac57b566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:59.490329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ec03-2db6-49e2-8db1-a19ac57b566c admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.490606 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ec03-2db6-49e2-8db1-a19ac57b566c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.490872 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ec03-2db6-49e2-8db1-a19ac57b566c admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.491495 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495ec03-2db6-49e2-8db1-a19ac57b566c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:59.493434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495ec03-2db6-49e2-8db1-a19ac57b566c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:59.515987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1219/2446] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 14:59:59 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 14:59:59.520265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282eefe-6275-4a31-b270-8bdbcf05845f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:59.532193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282eefe-6275-4a31-b270-8bdbcf05845f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:59.537229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b3fb8-1eb5-41d0-8f95-0fe93f7f283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:59.538884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b3fb8-1eb5-41d0-8f95-0fe93f7f283c admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.539171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b3fb8-1eb5-41d0-8f95-0fe93f7f283c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.539439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b3fb8-1eb5-41d0-8f95-0fe93f7f283c admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.540153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b3fb8-1eb5-41d0-8f95-0fe93f7f283c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:59.542331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b3fb8-1eb5-41d0-8f95-0fe93f7f283c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:59.553473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1228/2447] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 14:59:59.558231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ec03-2db6-49e2-8db1-a19ac57b566c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:59.568722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ec03-2db6-49e2-8db1-a19ac57b566c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:59.573273 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e5331-fa5c-4f3c-8e16-903c1b9569bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:59.573928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e5331-fa5c-4f3c-8e16-903c1b9569bc admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.574204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e5331-fa5c-4f3c-8e16-903c1b9569bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.574476 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e5331-fa5c-4f3c-8e16-903c1b9569bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.575115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e5331-fa5c-4f3c-8e16-903c1b9569bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:59.576731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e5331-fa5c-4f3c-8e16-903c1b9569bc admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:59.587667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1220/2448] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 14:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 14:59:59.592347 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b3fb8-1eb5-41d0-8f95-0fe93f7f283c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:59.603183 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b3fb8-1eb5-41d0-8f95-0fe93f7f283c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:59.607896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fba4b-1d65-4a77-a4c5-1330452294d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:59.608561 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fba4b-1d65-4a77-a4c5-1330452294d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.608876 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fba4b-1d65-4a77-a4c5-1330452294d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.609115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fba4b-1d65-4a77-a4c5-1330452294d4 admin admin] PATH_INFO: `/v3/projects/9605558553df4ab38128411033a5596e/users/513f2ecbbe1040baaff68c713b7b0e18/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.626337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241fba4b-1d65-4a77-a4c5-1330452294d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9605558553df4ab38128411033a5596e, user_id=513f2ecbbe1040baaff68c713b7b0e18, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:59.629101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241fba4b-1d65-4a77-a4c5-1330452294d4 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:59.641588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1229/2449] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 14:59:59 2022] PUT /identity/v3/projects/9605558553df4ab38128411033a5596e/users/513f2ecbbe1040baaff68c713b7b0e18/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 14:59:59.652154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb135f-33bd-4986-a462-63bf5ca36cc0 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.652615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb135f-33bd-4986-a462-63bf5ca36cc0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.652865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb135f-33bd-4986-a462-63bf5ca36cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.675143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:59.683662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-a3cb135f-33bd-4986-a462-63bf5ca36cc0 None None] MFA Rules not processed for user `513f2ecbbe1040baaff68c713b7b0e18`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 14:59:59.685463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a3cb135f-33bd-4986-a462-63bf5ca36cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:59.762933 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1221/2450] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 14:59:59.774426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fba4b-1d65-4a77-a4c5-1330452294d4 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 14:59:59.776210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-241fba4b-1d65-4a77-a4c5-1330452294d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 14:59:59.831437 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 14:59:59.837588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fba4b-1d65-4a77-a4c5-1330452294d4 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 14:59:59.842860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a570a-2075-482a-83d1-c0018b6cc706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 14:59:59.843573 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a570a-2075-482a-83d1-c0018b6cc706 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 14:59:59.844647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a570a-2075-482a-83d1-c0018b6cc706 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 14:59:59.845257 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a570a-2075-482a-83d1-c0018b6cc706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 14:59:59.854228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a570a-2075-482a-83d1-c0018b6cc706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 14:59:59.856084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a570a-2075-482a-83d1-c0018b6cc706 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 14:59:59.866676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1230/2451] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 14:59:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:00:00.804368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e556e-786e-4cdd-9689-7f64412ba452 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:00:00.804834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e556e-786e-4cdd-9689-7f64412ba452 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:00:00.804947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e556e-786e-4cdd-9689-7f64412ba452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:00:00.829727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-7f4e556e-786e-4cdd-9689-7f64412ba452 None None] MFA Rules not processed for user `89c747f18b434e469ae61755e9f639ca`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:00:00.830923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7f4e556e-786e-4cdd-9689-7f64412ba452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:00.883466 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1222/2452] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:00:05.911424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a570a-2075-482a-83d1-c0018b6cc706 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:00:05.913662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f44a570a-2075-482a-83d1-c0018b6cc706 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:06.020067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a570a-2075-482a-83d1-c0018b6cc706 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:00:06.025119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c2445-6f18-46e4-9d7f-ea96d0d9ad25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:00:06.025757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c2445-6f18-46e4-9d7f-ea96d0d9ad25 service placement] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:00:06.026036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c2445-6f18-46e4-9d7f-ea96d0d9ad25 service placement] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:00:06.026309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c2445-6f18-46e4-9d7f-ea96d0d9ad25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:00:06.028013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-076c2445-6f18-46e4-9d7f-ea96d0d9ad25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:06.040019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c2445-6f18-46e4-9d7f-ea96d0d9ad25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:00:06.041623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c2445-6f18-46e4-9d7f-ea96d0d9ad25 service placement] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:00:06.051310 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1231/2453] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:00:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:00:10.877003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=52 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:00:16.047737 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=45 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:00:23.730826 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e556e-786e-4cdd-9689-7f64412ba452 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:00:23.793771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e556e-786e-4cdd-9689-7f64412ba452 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:00:23.799243 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:00:23.799873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:00:23.800156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:00:23.800427 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] PATH_INFO: `/v3/users/513f2ecbbe1040baaff68c713b7b0e18` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:00:23.801700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=513f2ecbbe1040baaff68c713b7b0e18)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:00:23.803619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:00:23.850878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513f2ecbbe1040baaff68c713b7b0e18'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:00:23.851433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513f2ecbbe1040baaff68c713b7b0e18'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:00:23.858419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513f2ecbbe1040baaff68c713b7b0e18'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:00:23.867052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513f2ecbbe1040baaff68c713b7b0e18'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:00:23.867773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513f2ecbbe1040baaff68c713b7b0e18'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:00:23.885913 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1223/2454] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:00:23 2022] DELETE /identity/v3/users/513f2ecbbe1040baaff68c713b7b0e18 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:00:23.890425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa87260-da7a-4531-b2b5-1dc41dc3aeff None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:00:23.891318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa87260-da7a-4531-b2b5-1dc41dc3aeff None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:00:23.891769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa87260-da7a-4531-b2b5-1dc41dc3aeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:00:23.912696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-7aa87260-da7a-4531-b2b5-1dc41dc3aeff None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:00:23.913954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7aa87260-da7a-4531-b2b5-1dc41dc3aeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:23.973620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1232/2455] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:00:23.983690 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:00:23.985164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:24.075536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6137b1f-6dac-45ef-b5e9-f34ebb309612 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:00:24.079575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfe86e-4fa1-424f-8485-7527d70ac792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:00:24.080309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfe86e-4fa1-424f-8485-7527d70ac792 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:00:24.080590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfe86e-4fa1-424f-8485-7527d70ac792 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:00:24.080858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfe86e-4fa1-424f-8485-7527d70ac792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:00:24.090343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfe86e-4fa1-424f-8485-7527d70ac792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:00:24.091784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfe86e-4fa1-424f-8485-7527d70ac792 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:00:24.101886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1224/2456] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:00:24.382899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa87260-da7a-4531-b2b5-1dc41dc3aeff None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:00:24.384717 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7aa87260-da7a-4531-b2b5-1dc41dc3aeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:24.398834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa87260-da7a-4531-b2b5-1dc41dc3aeff None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:00:24.409638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:00:24.410381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:00:24.410729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:00:24.411072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] PATH_INFO: `/v3/projects/9605558553df4ab38128411033a5596e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:00:24.412272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9605558553df4ab38128411033a5596e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:00:24.422322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:00:24.475487 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9605558553df4ab38128411033a5596e', 'request_id': 'req-e0dd9797-c01a-4150-b16a-843e8fb54c4d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:00:24.485039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9605558553df4ab38128411033a5596e', 'request_id': 'req-e0dd9797-c01a-4150-b16a-843e8fb54c4d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:00:24.493406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9605558553df4ab38128411033a5596e', 'request_id': 'req-e0dd9797-c01a-4150-b16a-843e8fb54c4d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:00:24.504306 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] The token cache is being invalidate because project 9605558553df4ab38128411033a5596e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:00:24.504591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:00:24.505242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e0dd9797-c01a-4150-b16a-843e8fb54c4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:00:24.506771 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1233/2457] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:00:24 2022] DELETE /identity/v3/projects/9605558553df4ab38128411033a5596e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:00:34.097425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:00:34.130489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfe86e-4fa1-424f-8485-7527d70ac792 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:00:34.131868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-16bfe86e-4fa1-424f-8485-7527d70ac792 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:34.209019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfe86e-4fa1-424f-8485-7527d70ac792 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:00:34.255378 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9745d2-35a3-4896-8603-f9e04e18325d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:00:34.255982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9745d2-35a3-4896-8603-f9e04e18325d service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:00:34.256258 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9745d2-35a3-4896-8603-f9e04e18325d service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:00:34.256527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9745d2-35a3-4896-8603-f9e04e18325d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:00:34.258211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3c9745d2-35a3-4896-8603-f9e04e18325d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:34.316760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9745d2-35a3-4896-8603-f9e04e18325d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:00:34.318242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9745d2-35a3-4896-8603-f9e04e18325d service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:00:34.327197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1225/2458] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:00:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:00:34.501279 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:00:44.322990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=77 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:00:46.398157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc942f2a-7fe8-4809-aa6d-7c8fe7925440 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:00:46.398652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc942f2a-7fe8-4809-aa6d-7c8fe7925440 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:00:46.398759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc942f2a-7fe8-4809-aa6d-7c8fe7925440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:00:46.417112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-cc942f2a-7fe8-4809-aa6d-7c8fe7925440 None None] MFA Rules not processed for user `7cfae014351641d8a949775875a45958`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:00:46.418330 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-cc942f2a-7fe8-4809-aa6d-7c8fe7925440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:46.425484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1234/2459] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:00:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:00:56.412219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:00:56.787057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b18647-9fa0-441f-a1d7-e8082ee66175 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:00:56.787555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b18647-9fa0-441f-a1d7-e8082ee66175 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:00:56.787668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b18647-9fa0-441f-a1d7-e8082ee66175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:00:56.816894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-14b18647-9fa0-441f-a1d7-e8082ee66175 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:00:56.818118 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-14b18647-9fa0-441f-a1d7-e8082ee66175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:56.879106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1226/2460] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 15:00:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:00:56.927268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc942f2a-7fe8-4809-aa6d-7c8fe7925440 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:00:56.928885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-cc942f2a-7fe8-4809-aa6d-7c8fe7925440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:00:57.005809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc942f2a-7fe8-4809-aa6d-7c8fe7925440 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:00:57.010752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7df40-35e4-4a80-b0ab-2201db9cb1c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:00:57.011358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7df40-35e4-4a80-b0ab-2201db9cb1c5 service swift] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:00:57.011637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7df40-35e4-4a80-b0ab-2201db9cb1c5 service swift] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:00:57.011912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7df40-35e4-4a80-b0ab-2201db9cb1c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:00:57.020286 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7df40-35e4-4a80-b0ab-2201db9cb1c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:00:57.021939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7df40-35e4-4a80-b0ab-2201db9cb1c5 service swift] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:00:57.029779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1235/2461] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 15:00:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:01:05.458034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b18647-9fa0-441f-a1d7-e8082ee66175 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:05.459447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-14b18647-9fa0-441f-a1d7-e8082ee66175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:05.522941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b18647-9fa0-441f-a1d7-e8082ee66175 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:05.526816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db32ed-6fa4-410e-8913-8efb5a0d3943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:05.527445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db32ed-6fa4-410e-8913-8efb5a0d3943 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:05.527729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db32ed-6fa4-410e-8913-8efb5a0d3943 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:05.528136 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db32ed-6fa4-410e-8913-8efb5a0d3943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:05.529951 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-19db32ed-6fa4-410e-8913-8efb5a0d3943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:05.611171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db32ed-6fa4-410e-8913-8efb5a0d3943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:05.612553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db32ed-6fa4-410e-8913-8efb5a0d3943 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:05.623046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1227/2462] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:01:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:01:07.027187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=32 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:01:14.722585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2422782-ec3c-47aa-b495-4c047196134a None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:14.723206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2422782-ec3c-47aa-b495-4c047196134a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:14.723206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2422782-ec3c-47aa-b495-4c047196134a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:14.743439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-b2422782-ec3c-47aa-b495-4c047196134a None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:01:14.744680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b2422782-ec3c-47aa-b495-4c047196134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:14.752608 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1236/2463] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 15:01:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:01:14.764515 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db32ed-6fa4-410e-8913-8efb5a0d3943 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:14.776288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db32ed-6fa4-410e-8913-8efb5a0d3943 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:14.781214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567980c-30c2-4c7a-a6cd-5985afbccd95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:14.781880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567980c-30c2-4c7a-a6cd-5985afbccd95 service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:14.782176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567980c-30c2-4c7a-a6cd-5985afbccd95 service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:14.782444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567980c-30c2-4c7a-a6cd-5985afbccd95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:14.790332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567980c-30c2-4c7a-a6cd-5985afbccd95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:14.791920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567980c-30c2-4c7a-a6cd-5985afbccd95 service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:14.800111 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1228/2464] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 15:01:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:01:24.736920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:01:24.796739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=88 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:01:31.069846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2422782-ec3c-47aa-b495-4c047196134a None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:31.080178 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2422782-ec3c-47aa-b495-4c047196134a None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:31.084021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8da776-6adb-4508-8269-2e573e51003f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:31.084964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8da776-6adb-4508-8269-2e573e51003f service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:31.084964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8da776-6adb-4508-8269-2e573e51003f service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:31.085321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8da776-6adb-4508-8269-2e573e51003f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:31.087724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-fa8da776-6adb-4508-8269-2e573e51003f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:31.146437 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8da776-6adb-4508-8269-2e573e51003f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:31.147879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8da776-6adb-4508-8269-2e573e51003f service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:31.156894 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1237/2465] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:01:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:01:37.957250 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44cf1-7417-4ec1-a388-964423aa4a43 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:37.957787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44cf1-7417-4ec1-a388-964423aa4a43 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:37.957893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44cf1-7417-4ec1-a388-964423aa4a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:37.981437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-78c44cf1-7417-4ec1-a388-964423aa4a43 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:01:37.982772 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-78c44cf1-7417-4ec1-a388-964423aa4a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:37.990753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1229/2466] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 3 15:01:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:01:38.003087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8da776-6adb-4508-8269-2e573e51003f service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:38.014423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8da776-6adb-4508-8269-2e573e51003f service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:38.019362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fc8ce-107e-4cc1-ba00-4382aa6a025e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:38.019964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fc8ce-107e-4cc1-ba00-4382aa6a025e service swift] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:38.020241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fc8ce-107e-4cc1-ba00-4382aa6a025e service swift] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:38.020512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fc8ce-107e-4cc1-ba00-4382aa6a025e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:38.028464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fc8ce-107e-4cc1-ba00-4382aa6a025e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:38.030106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fc8ce-107e-4cc1-ba00-4382aa6a025e service swift] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:38.038582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1238/2467] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 15:01:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:01:39.213966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c44cf1-7417-4ec1-a388-964423aa4a43 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:39.215420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-78c44cf1-7417-4ec1-a388-964423aa4a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:39.227293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c44cf1-7417-4ec1-a388-964423aa4a43 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:39.230943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777e0b5-0439-40f5-a4c4-5085293fcdb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:39.231537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e0b5-0439-40f5-a4c4-5085293fcdb4 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:39.231821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e0b5-0439-40f5-a4c4-5085293fcdb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:39.232093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e0b5-0439-40f5-a4c4-5085293fcdb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:39.233769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6777e0b5-0439-40f5-a4c4-5085293fcdb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:39.299533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777e0b5-0439-40f5-a4c4-5085293fcdb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:39.301020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777e0b5-0439-40f5-a4c4-5085293fcdb4 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:39.311107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1230/2468] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:01:42.119453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fc8ce-107e-4cc1-ba00-4382aa6a025e service swift] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:42.121108 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f74fc8ce-107e-4cc1-ba00-4382aa6a025e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:42.221647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fc8ce-107e-4cc1-ba00-4382aa6a025e service swift] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:42.226910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dfc90-3d21-45fb-ac19-aefc014bae97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:42.227524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dfc90-3d21-45fb-ac19-aefc014bae97 service placement] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:42.227802 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dfc90-3d21-45fb-ac19-aefc014bae97 service placement] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:42.228190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dfc90-3d21-45fb-ac19-aefc014bae97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:42.229919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9a9dfc90-3d21-45fb-ac19-aefc014bae97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:42.242714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dfc90-3d21-45fb-ac19-aefc014bae97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:42.244381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dfc90-3d21-45fb-ac19-aefc014bae97 service placement] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:42.254710 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1239/2469] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:01:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:01:45.215788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777e0b5-0439-40f5-a4c4-5085293fcdb4 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:45.228735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777e0b5-0439-40f5-a4c4-5085293fcdb4 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:45.234103 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cb989-a557-43b5-b17b-4a59a8ab8cd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:45.235525 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cb989-a557-43b5-b17b-4a59a8ab8cd2 service placement] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:45.236150 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cb989-a557-43b5-b17b-4a59a8ab8cd2 service placement] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:45.236465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cb989-a557-43b5-b17b-4a59a8ab8cd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:45.238488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ac3cb989-a557-43b5-b17b-4a59a8ab8cd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:45.251376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cb989-a557-43b5-b17b-4a59a8ab8cd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:45.253635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cb989-a557-43b5-b17b-4a59a8ab8cd2 service placement] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:45.270005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1231/2470] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:01:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:01:45.276612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dfc90-3d21-45fb-ac19-aefc014bae97 service placement] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:45.278871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9a9dfc90-3d21-45fb-ac19-aefc014bae97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:45.298124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dfc90-3d21-45fb-ac19-aefc014bae97 service placement] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:45.303196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:45.303736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:45.304001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:45.304428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] PATH_INFO: `/v3/users/7725c72e89964487843bc9bd46b8e3d2` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:45.305570 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7725c72e89964487843bc9bd46b8e3d2)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:45.307488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:45.356100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7725c72e89964487843bc9bd46b8e3d2'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:45.356754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7725c72e89964487843bc9bd46b8e3d2'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:45.364411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7725c72e89964487843bc9bd46b8e3d2'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:45.374535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:01:45.377007 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7725c72e89964487843bc9bd46b8e3d2'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:45.387436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7725c72e89964487843bc9bd46b8e3d2'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:45.394034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1240/2471] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:01:45 2022] DELETE /identity/v3/users/7725c72e89964487843bc9bd46b8e3d2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:01:45.792621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cb989-a557-43b5-b17b-4a59a8ab8cd2 service placement] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:45.803626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cb989-a557-43b5-b17b-4a59a8ab8cd2 service placement] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:45.861582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:45.862213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:45.862493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:45.862766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] PATH_INFO: `/v3/projects/8fae62571c3c402a8fcc4afaf4268310` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:45.863934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fae62571c3c402a8fcc4afaf4268310)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:45.865865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:45.911706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fae62571c3c402a8fcc4afaf4268310', 'request_id': 'req-aa62ad99-f59f-4dee-ad60-271edcd4561d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:45.919383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fae62571c3c402a8fcc4afaf4268310', 'request_id': 'req-aa62ad99-f59f-4dee-ad60-271edcd4561d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:45.925519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fae62571c3c402a8fcc4afaf4268310', 'request_id': 'req-aa62ad99-f59f-4dee-ad60-271edcd4561d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:45.936530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] The token cache is being invalidate because project 8fae62571c3c402a8fcc4afaf4268310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:01:45.936827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:45.937465 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:45.939483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1232/2472] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:01:45 2022] DELETE /identity/v3/projects/8fae62571c3c402a8fcc4afaf4268310 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:01:45.943491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:45.944983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:45.959012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8724045-10cc-4a2d-97e7-1a7b5126d788 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:46.020224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:46.020503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:46.020791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:46.021067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] PATH_INFO: `/v3/users/21b1d77c6794409e98576eb207a843cb` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:46.022809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] RBAC: Authorizing `identity:delete_user(user_id=21b1d77c6794409e98576eb207a843cb)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:46.024983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:46.084418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21b1d77c6794409e98576eb207a843cb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:46.085166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21b1d77c6794409e98576eb207a843cb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:46.093634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21b1d77c6794409e98576eb207a843cb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:46.108159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21b1d77c6794409e98576eb207a843cb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:46.118091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21b1d77c6794409e98576eb207a843cb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:46.124950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1241/2473] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:01:45 2022] DELETE /identity/v3/users/21b1d77c6794409e98576eb207a843cb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:01:46.316674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:46.318988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:46.335888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:01:46.340828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62ad99-f59f-4dee-ad60-271edcd4561d admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:46.408841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:46.409477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:46.409794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:46.410071 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] PATH_INFO: `/v3/projects/56b805f66853429593a3f0b37b907b3f` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:46.411190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b805f66853429593a3f0b37b907b3f)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:46.413128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:46.460589 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b805f66853429593a3f0b37b907b3f', 'request_id': 'req-4215eda9-1f50-4a04-9b1c-517a444fc07d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:46.467742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b805f66853429593a3f0b37b907b3f', 'request_id': 'req-4215eda9-1f50-4a04-9b1c-517a444fc07d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:46.473946 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b805f66853429593a3f0b37b907b3f', 'request_id': 'req-4215eda9-1f50-4a04-9b1c-517a444fc07d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:46.483937 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] The token cache is being invalidate because project 56b805f66853429593a3f0b37b907b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:01:46.484220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:46.485325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:01:46.486821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1233/2474] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:01:46 2022] DELETE /identity/v3/projects/56b805f66853429593a3f0b37b907b3f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:01:56.121213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:DELETE=12,keystone:SELECT=56,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:01:56.481070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:01:56.694131 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:01:56.695512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:56.751381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38c942-18c6-4e84-a8de-e77ac869d80d admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:01:56.812403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea727e45-a197-49cf-b4dd-c34436c478fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:01:56.813346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea727e45-a197-49cf-b4dd-c34436c478fd service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:01:56.813724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea727e45-a197-49cf-b4dd-c34436c478fd service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:01:56.814041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea727e45-a197-49cf-b4dd-c34436c478fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:01:56.816215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ea727e45-a197-49cf-b4dd-c34436c478fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:01:56.897142 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea727e45-a197-49cf-b4dd-c34436c478fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:01:56.898664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea727e45-a197-49cf-b4dd-c34436c478fd service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:01:56.920115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1242/2475] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:01:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:05.017827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:05.019244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:05.031471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215eda9-1f50-4a04-9b1c-517a444fc07d admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:05.034970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07a6d8-2411-4ed1-8eb9-496f662518fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:05.035554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a6d8-2411-4ed1-8eb9-496f662518fe service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:05.035824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a6d8-2411-4ed1-8eb9-496f662518fe service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:05.036089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a6d8-2411-4ed1-8eb9-496f662518fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:05.037709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2d07a6d8-2411-4ed1-8eb9-496f662518fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:05.050756 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07a6d8-2411-4ed1-8eb9-496f662518fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:05.052160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07a6d8-2411-4ed1-8eb9-496f662518fe service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:05.062745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1234/2476] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:06.917462 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=83 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:15.058201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=6 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:26.945633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea727e45-a197-49cf-b4dd-c34436c478fd service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:26.947024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ea727e45-a197-49cf-b4dd-c34436c478fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:27.001920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea727e45-a197-49cf-b4dd-c34436c478fd service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:27.005829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f046-791b-42a4-b7d4-8981d0d20ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:27.006431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f046-791b-42a4-b7d4-8981d0d20ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:27.006709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f046-791b-42a4-b7d4-8981d0d20ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:27.006976 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f046-791b-42a4-b7d4-8981d0d20ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:27.008673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b730f046-791b-42a4-b7d4-8981d0d20ef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:27.071515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730f046-791b-42a4-b7d4-8981d0d20ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:27.072945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730f046-791b-42a4-b7d4-8981d0d20ef6 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:27.082796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1243/2477] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:02:27.255372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07a6d8-2411-4ed1-8eb9-496f662518fe service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:27.256771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2d07a6d8-2411-4ed1-8eb9-496f662518fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:27.267966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07a6d8-2411-4ed1-8eb9-496f662518fe service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:27.271481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcc443-7f9f-43bb-a8c8-5a5a34121c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:27.272053 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcc443-7f9f-43bb-a8c8-5a5a34121c3f service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:27.272332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcc443-7f9f-43bb-a8c8-5a5a34121c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:27.272601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcc443-7f9f-43bb-a8c8-5a5a34121c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:27.274272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-08bcc443-7f9f-43bb-a8c8-5a5a34121c3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:27.286709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcc443-7f9f-43bb-a8c8-5a5a34121c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:27.288144 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcc443-7f9f-43bb-a8c8-5a5a34121c3f service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:27.299050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1235/2478] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:30.869568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f046-791b-42a4-b7d4-8981d0d20ef6 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:30.871283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b730f046-791b-42a4-b7d4-8981d0d20ef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:30.885102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f046-791b-42a4-b7d4-8981d0d20ef6 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:30.889912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:30.890623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:30.890977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:30.891346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] PATH_INFO: `/v3/users/0be78e1abb3947a2ac5881b9fd2f6fae` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:30.907364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be78e1abb3947a2ac5881b9fd2f6fae)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:30.909627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:30.993057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be78e1abb3947a2ac5881b9fd2f6fae'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:30.995017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be78e1abb3947a2ac5881b9fd2f6fae'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.005649 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be78e1abb3947a2ac5881b9fd2f6fae'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.029068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be78e1abb3947a2ac5881b9fd2f6fae'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.038652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be78e1abb3947a2ac5881b9fd2f6fae'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.046909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1244/2479] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:30 2022] DELETE /identity/v3/users/0be78e1abb3947a2ac5881b9fd2f6fae => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:31.258467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcc443-7f9f-43bb-a8c8-5a5a34121c3f service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.259825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-08bcc443-7f9f-43bb-a8c8-5a5a34121c3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:31.265860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.267250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:31.284760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcc443-7f9f-43bb-a8c8-5a5a34121c3f service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.305168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:31.357782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.358575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.358844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.359111 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] PATH_INFO: `/v3/users/89c747f18b434e469ae61755e9f639ca` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.371873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625f2a6-9ede-414f-87cc-50afdf72274f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.373634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] RBAC: Authorizing `identity:delete_user(user_id=89c747f18b434e469ae61755e9f639ca)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:31.375651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:31.377149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.378141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.378601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.379047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] PATH_INFO: `/v3/projects/8cf070b7d6ff419392acaddbbee1e242` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.394391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cf070b7d6ff419392acaddbbee1e242)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:31.399456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:31.433413 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89c747f18b434e469ae61755e9f639ca'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.434095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89c747f18b434e469ae61755e9f639ca'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.442953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89c747f18b434e469ae61755e9f639ca'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.449312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cf070b7d6ff419392acaddbbee1e242', 'request_id': 'req-2be7ddca-f0ea-4786-a742-c4378e0b02d1'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.450650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89c747f18b434e469ae61755e9f639ca'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.451363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-bc256b8f-c424-45c9-b3e3-c7a314fae02c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89c747f18b434e469ae61755e9f639ca'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.458363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cf070b7d6ff419392acaddbbee1e242', 'request_id': 'req-2be7ddca-f0ea-4786-a742-c4378e0b02d1'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.465634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cf070b7d6ff419392acaddbbee1e242', 'request_id': 'req-2be7ddca-f0ea-4786-a742-c4378e0b02d1'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.471205 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1236/2480] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:31 2022] DELETE /identity/v3/users/89c747f18b434e469ae61755e9f639ca => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:31.479587 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] The token cache is being invalidate because project 8cf070b7d6ff419392acaddbbee1e242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:31.479882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.480528 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2be7ddca-f0ea-4786-a742-c4378e0b02d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.482049 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1245/2481] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:31 2022] DELETE /identity/v3/projects/8cf070b7d6ff419392acaddbbee1e242 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:31.486981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d321-2eb1-4ee5-8676-f9303040d8e8 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.487403 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d321-2eb1-4ee5-8676-f9303040d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.487689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d321-2eb1-4ee5-8676-f9303040d8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.507473 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-6785d321-2eb1-4ee5-8676-f9303040d8e8 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:31.508620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6785d321-2eb1-4ee5-8676-f9303040d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:31.566201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:31.581163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1237/2482] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:31.590429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedf378-57e1-445a-9dc3-29b10f650e33 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.590838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedf378-57e1-445a-9dc3-29b10f650e33 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.591190 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedf378-57e1-445a-9dc3-29b10f650e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.611412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ceedf378-57e1-445a-9dc3-29b10f650e33 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:31.612712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ceedf378-57e1-445a-9dc3-29b10f650e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:31.620769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1246/2483] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:31.625584 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785d321-2eb1-4ee5-8676-f9303040d8e8 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.638147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785d321-2eb1-4ee5-8676-f9303040d8e8 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.644460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddab6a-3574-4ad9-80e5-3dd542d8b743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.645071 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddab6a-3574-4ad9-80e5-3dd542d8b743 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.645352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddab6a-3574-4ad9-80e5-3dd542d8b743 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.645657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddab6a-3574-4ad9-80e5-3dd542d8b743 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.646421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddab6a-3574-4ad9-80e5-3dd542d8b743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:31.647000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddab6a-3574-4ad9-80e5-3dd542d8b743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:31.648610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddab6a-3574-4ad9-80e5-3dd542d8b743 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:31.654035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedf378-57e1-445a-9dc3-29b10f650e33 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.657099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ceedf378-57e1-445a-9dc3-29b10f650e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:31.663651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1238/2484] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:02:31.667778 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddab6a-3574-4ad9-80e5-3dd542d8b743 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.671605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedf378-57e1-445a-9dc3-29b10f650e33 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.677747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddab6a-3574-4ad9-80e5-3dd542d8b743 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.678412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.679201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.679623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.680034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] PATH_INFO: `/v3/projects/f441c0592cd146d3be851ac96cbc49dc` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.683388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc23c38-1be9-4977-9954-ee9a45deecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.683985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc23c38-1be9-4977-9954-ee9a45deecc8 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.684266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc23c38-1be9-4977-9954-ee9a45deecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.684566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc23c38-1be9-4977-9954-ee9a45deecc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.685206 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc23c38-1be9-4977-9954-ee9a45deecc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:31.687658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc23c38-1be9-4977-9954-ee9a45deecc8 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:31.691993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f441c0592cd146d3be851ac96cbc49dc)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:31.694037 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:31.730292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1239/2485] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:31 2022] POST /identity/v3/projects => generated 405 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:31.731370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc23c38-1be9-4977-9954-ee9a45deecc8 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.744061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f441c0592cd146d3be851ac96cbc49dc', 'request_id': 'req-76a2756b-8c0e-437b-b3e4-48c272997362'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.752595 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f441c0592cd146d3be851ac96cbc49dc', 'request_id': 'req-76a2756b-8c0e-437b-b3e4-48c272997362'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.760011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f441c0592cd146d3be851ac96cbc49dc', 'request_id': 'req-76a2756b-8c0e-437b-b3e4-48c272997362'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.768026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:31.772498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] The token cache is being invalidate because project f441c0592cd146d3be851ac96cbc49dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:31.772969 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.774149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-76a2756b-8c0e-437b-b3e4-48c272997362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:31.777572 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1247/2486] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:31 2022] DELETE /identity/v3/projects/f441c0592cd146d3be851ac96cbc49dc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:31.780538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000e1a-0cb1-451a-b370-a802d6ff207f None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.780823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000e1a-0cb1-451a-b370-a802d6ff207f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.781101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000e1a-0cb1-451a-b370-a802d6ff207f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.799294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc23c38-1be9-4977-9954-ee9a45deecc8 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.800461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-96000e1a-0cb1-451a-b370-a802d6ff207f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:31.801708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-96000e1a-0cb1-451a-b370-a802d6ff207f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:31.858678 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc53187-a2bb-4006-b329-111326d33ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.859283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc53187-a2bb-4006-b329-111326d33ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.859560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc53187-a2bb-4006-b329-111326d33ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.859830 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc53187-a2bb-4006-b329-111326d33ce1 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.860463 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc53187-a2bb-4006-b329-111326d33ce1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:31.861487 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1248/2487] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:31.862452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc53187-a2bb-4006-b329-111326d33ce1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:31.869879 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a34e0-6f14-43be-bf54-0dcb6144081e None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.870378 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a34e0-6f14-43be-bf54-0dcb6144081e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.870461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a34e0-6f14-43be-bf54-0dcb6144081e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.886464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1240/2488] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:31 2022] POST /identity/v3/users => generated 303 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:31.890810 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc53187-a2bb-4006-b329-111326d33ce1 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.892457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1bc53187-a2bb-4006-b329-111326d33ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:31.895996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-4e4a34e0-6f14-43be-bf54-0dcb6144081e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:31.897618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4e4a34e0-6f14-43be-bf54-0dcb6144081e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:31.904195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc53187-a2bb-4006-b329-111326d33ce1 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.905920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1249/2489] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:31.909051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede509d-a6e3-4b24-a31a-184fa02782a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.909684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede509d-a6e3-4b24-a31a-184fa02782a2 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.909828 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a34e0-6f14-43be-bf54-0dcb6144081e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.909984 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede509d-a6e3-4b24-a31a-184fa02782a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.910228 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede509d-a6e3-4b24-a31a-184fa02782a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.910867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede509d-a6e3-4b24-a31a-184fa02782a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:31.912451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede509d-a6e3-4b24-a31a-184fa02782a2 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:31.920444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a34e0-6f14-43be-bf54-0dcb6144081e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.923022 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1241/2490] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:31.926052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f65d2-4b74-4da5-822c-89c8dd122e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.926746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f65d2-4b74-4da5-822c-89c8dd122e35 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.926888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede509d-a6e3-4b24-a31a-184fa02782a2 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.927212 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f65d2-4b74-4da5-822c-89c8dd122e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.927452 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f65d2-4b74-4da5-822c-89c8dd122e35 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.928117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f65d2-4b74-4da5-822c-89c8dd122e35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:31.928426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f65d2-4b74-4da5-822c-89c8dd122e35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:31.930074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f65d2-4b74-4da5-822c-89c8dd122e35 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:31.937254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede509d-a6e3-4b24-a31a-184fa02782a2 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.943008 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee066-5517-4a97-9c1b-b919e1c54441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.943693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee066-5517-4a97-9c1b-b919e1c54441 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.943973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee066-5517-4a97-9c1b-b919e1c54441 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.944285 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee066-5517-4a97-9c1b-b919e1c54441 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.944887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee066-5517-4a97-9c1b-b919e1c54441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:31.946516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee066-5517-4a97-9c1b-b919e1c54441 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:31.947241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1250/2491] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:02:31.948120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f65d2-4b74-4da5-822c-89c8dd122e35 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.958626 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1242/2492] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:31.959591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f65d2-4b74-4da5-822c-89c8dd122e35 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.962430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee066-5517-4a97-9c1b-b919e1c54441 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:31.964388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41016333-a67a-4b6e-af2d-9c0e84d2bb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.964992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41016333-a67a-4b6e-af2d-9c0e84d2bb53 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.965276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41016333-a67a-4b6e-af2d-9c0e84d2bb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.965580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41016333-a67a-4b6e-af2d-9c0e84d2bb53 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.966216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41016333-a67a-4b6e-af2d-9c0e84d2bb53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:31.969051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41016333-a67a-4b6e-af2d-9c0e84d2bb53 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:31.972123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee066-5517-4a97-9c1b-b919e1c54441 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:31.977211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704b565-8b28-4b5e-9212-5a1d66007b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:31.977876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704b565-8b28-4b5e-9212-5a1d66007b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:31.978152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704b565-8b28-4b5e-9212-5a1d66007b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:31.978422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704b565-8b28-4b5e-9212-5a1d66007b84 admin admin] PATH_INFO: `/v3/projects/adc94ec120244542bac3f79a50b96c8a/users/aa2186ef8f8e4312b94c2dc57787eb93/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:31.997157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704b565-8b28-4b5e-9212-5a1d66007b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adc94ec120244542bac3f79a50b96c8a, user_id=aa2186ef8f8e4312b94c2dc57787eb93, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.000863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1251/2493] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:31 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:32.001147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704b565-8b28-4b5e-9212-5a1d66007b84 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.004215 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41016333-a67a-4b6e-af2d-9c0e84d2bb53 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.018850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1243/2494] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:31 2022] PUT /identity/v3/projects/adc94ec120244542bac3f79a50b96c8a/users/aa2186ef8f8e4312b94c2dc57787eb93/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:32.031440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305e093-a513-4c30-a757-5c07d631ebf9 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.033749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305e093-a513-4c30-a757-5c07d631ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.034126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305e093-a513-4c30-a757-5c07d631ebf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.044622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:32.073838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5305e093-a513-4c30-a757-5c07d631ebf9 None None] MFA Rules not processed for user `aa2186ef8f8e4312b94c2dc57787eb93`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:32.075118 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5305e093-a513-4c30-a757-5c07d631ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:32.087536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41016333-a67a-4b6e-af2d-9c0e84d2bb53 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.148728 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1244/2495] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:32 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:32.152746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305e093-a513-4c30-a757-5c07d631ebf9 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.160126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e44e54-5ddf-4951-b963-3220abccc122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.160743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e44e54-5ddf-4951-b963-3220abccc122 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.161029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e44e54-5ddf-4951-b963-3220abccc122 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.161299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e44e54-5ddf-4951-b963-3220abccc122 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.161967 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e44e54-5ddf-4951-b963-3220abccc122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.163907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e44e54-5ddf-4951-b963-3220abccc122 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.178066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:32.191080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1252/2496] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:32 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:32.195030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e44e54-5ddf-4951-b963-3220abccc122 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.205409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e44e54-5ddf-4951-b963-3220abccc122 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.210394 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71496de8-10f9-409c-9f04-14914c8b797c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.211021 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71496de8-10f9-409c-9f04-14914c8b797c admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.211289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71496de8-10f9-409c-9f04-14914c8b797c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.211807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71496de8-10f9-409c-9f04-14914c8b797c admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.212216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71496de8-10f9-409c-9f04-14914c8b797c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.213912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71496de8-10f9-409c-9f04-14914c8b797c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.224040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1253/2497] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:32.228011 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305e093-a513-4c30-a757-5c07d631ebf9 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.229432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71496de8-10f9-409c-9f04-14914c8b797c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.232672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61a7a0-6d8f-4676-8229-bbfa2a524328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.233269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61a7a0-6d8f-4676-8229-bbfa2a524328 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.234092 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61a7a0-6d8f-4676-8229-bbfa2a524328 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.234357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61a7a0-6d8f-4676-8229-bbfa2a524328 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.234979 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61a7a0-6d8f-4676-8229-bbfa2a524328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.236862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61a7a0-6d8f-4676-8229-bbfa2a524328 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.239332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71496de8-10f9-409c-9f04-14914c8b797c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.244253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a12d4-eb9c-41a0-9324-e0612eff3d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.244836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a12d4-eb9c-41a0-9324-e0612eff3d05 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.245120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a12d4-eb9c-41a0-9324-e0612eff3d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.245407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a12d4-eb9c-41a0-9324-e0612eff3d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.246482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a12d4-eb9c-41a0-9324-e0612eff3d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.248078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a12d4-eb9c-41a0-9324-e0612eff3d05 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.259247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1254/2498] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:32.266221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1245/2499] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:32 2022] POST /identity/v3/projects => generated 405 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:32.266707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a12d4-eb9c-41a0-9324-e0612eff3d05 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.267307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61a7a0-6d8f-4676-8229-bbfa2a524328 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.273417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a12d4-eb9c-41a0-9324-e0612eff3d05 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.333821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61a7a0-6d8f-4676-8229-bbfa2a524328 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.335034 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05686f8e-c630-4381-8012-88ada7402464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.335712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05686f8e-c630-4381-8012-88ada7402464 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.335997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05686f8e-c630-4381-8012-88ada7402464 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.336275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05686f8e-c630-4381-8012-88ada7402464 admin admin] PATH_INFO: `/v3/projects/8203ec601e074484b26de4380c9c9257/users/9d36c4b97e6b4df780ee04bc02981a30/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.338468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74ea9c-5b43-49a9-8bf9-3d41a31be979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.339066 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74ea9c-5b43-49a9-8bf9-3d41a31be979 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.339339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74ea9c-5b43-49a9-8bf9-3d41a31be979 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.339600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74ea9c-5b43-49a9-8bf9-3d41a31be979 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.340218 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74ea9c-5b43-49a9-8bf9-3d41a31be979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.342222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74ea9c-5b43-49a9-8bf9-3d41a31be979 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.350823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05686f8e-c630-4381-8012-88ada7402464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8203ec601e074484b26de4380c9c9257, user_id=9d36c4b97e6b4df780ee04bc02981a30, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.353532 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05686f8e-c630-4381-8012-88ada7402464 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.365505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1246/2500] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:32 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:32.365938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1255/2501] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:32 2022] PUT /identity/v3/projects/8203ec601e074484b26de4380c9c9257/users/9d36c4b97e6b4df780ee04bc02981a30/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:32.369448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74ea9c-5b43-49a9-8bf9-3d41a31be979 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.376459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a2ff-ae03-4ed5-a735-bc4cf9d0b3eb None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.376750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a2ff-ae03-4ed5-a735-bc4cf9d0b3eb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.377030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a2ff-ae03-4ed5-a735-bc4cf9d0b3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.379582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74ea9c-5b43-49a9-8bf9-3d41a31be979 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.383452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:32.404265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-91b4a2ff-ae03-4ed5-a735-bc4cf9d0b3eb None None] MFA Rules not processed for user `9d36c4b97e6b4df780ee04bc02981a30`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:32.405675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-91b4a2ff-ae03-4ed5-a735-bc4cf9d0b3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:32.438798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8187e-2416-46ff-8701-42e775211561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.439401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8187e-2416-46ff-8701-42e775211561 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.439675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8187e-2416-46ff-8701-42e775211561 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.439947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8187e-2416-46ff-8701-42e775211561 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.440591 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8187e-2416-46ff-8701-42e775211561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.442229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8187e-2416-46ff-8701-42e775211561 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.451753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1247/2502] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:32.456204 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8187e-2416-46ff-8701-42e775211561 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.465588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8187e-2416-46ff-8701-42e775211561 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.470557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d5591-44d6-46c1-abaf-4ab17ac0b8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.471178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d5591-44d6-46c1-abaf-4ab17ac0b8c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.471288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1256/2503] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:32.471456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d5591-44d6-46c1-abaf-4ab17ac0b8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.471732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d5591-44d6-46c1-abaf-4ab17ac0b8c3 admin admin] PATH_INFO: `/v3/projects/a4ac0b9e1a9947948cbc2cbbe89b3ac2/users/2f7ca535a64546f7a9b4421dad46c484/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.482385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4a2ff-ae03-4ed5-a735-bc4cf9d0b3eb None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.484593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-91b4a2ff-ae03-4ed5-a735-bc4cf9d0b3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:32.487234 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d5591-44d6-46c1-abaf-4ab17ac0b8c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ac0b9e1a9947948cbc2cbbe89b3ac2, user_id=2f7ca535a64546f7a9b4421dad46c484, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.490023 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d5591-44d6-46c1-abaf-4ab17ac0b8c3 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.504052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1248/2504] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:32 2022] PUT /identity/v3/projects/a4ac0b9e1a9947948cbc2cbbe89b3ac2/users/2f7ca535a64546f7a9b4421dad46c484/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:32.507909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d5591-44d6-46c1-abaf-4ab17ac0b8c3 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.538465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:32.564758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4a2ff-ae03-4ed5-a735-bc4cf9d0b3eb None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.568159 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d5591-44d6-46c1-abaf-4ab17ac0b8c3 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.572870 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b4dbb-e7cf-475f-b612-c5646b08423a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.573934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b4dbb-e7cf-475f-b612-c5646b08423a admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.574334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b4dbb-e7cf-475f-b612-c5646b08423a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.574620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b4dbb-e7cf-475f-b612-c5646b08423a admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.575264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b4dbb-e7cf-475f-b612-c5646b08423a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.576934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b4dbb-e7cf-475f-b612-c5646b08423a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.586954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1249/2505] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:32.591026 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b4dbb-e7cf-475f-b612-c5646b08423a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.601310 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b4dbb-e7cf-475f-b612-c5646b08423a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.605975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0ee27-04d7-4c5a-a0f3-afb8ea8f5b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.606602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0ee27-04d7-4c5a-a0f3-afb8ea8f5b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.606882 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0ee27-04d7-4c5a-a0f3-afb8ea8f5b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.607163 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0ee27-04d7-4c5a-a0f3-afb8ea8f5b69 admin admin] PATH_INFO: `/v3/projects/a4ac0b9e1a9947948cbc2cbbe89b3ac2/users/2f7ca535a64546f7a9b4421dad46c484/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.609723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0ee27-04d7-4c5a-a0f3-afb8ea8f5b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ac0b9e1a9947948cbc2cbbe89b3ac2, user_id=2f7ca535a64546f7a9b4421dad46c484, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.612366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0ee27-04d7-4c5a-a0f3-afb8ea8f5b69 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.622658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1250/2506] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:32 2022] PUT /identity/v3/projects/a4ac0b9e1a9947948cbc2cbbe89b3ac2/users/2f7ca535a64546f7a9b4421dad46c484/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:32.633280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac57cd-6865-4f65-b368-45de0b1fc4b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.633921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac57cd-6865-4f65-b368-45de0b1fc4b3 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.634033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bed889-3620-4168-98f6-44df8c94ed04 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.634200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac57cd-6865-4f65-b368-45de0b1fc4b3 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.634309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bed889-3620-4168-98f6-44df8c94ed04 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.634467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac57cd-6865-4f65-b368-45de0b1fc4b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.634571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bed889-3620-4168-98f6-44df8c94ed04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.662254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f6bed889-3620-4168-98f6-44df8c94ed04 None None] MFA Rules not processed for user `2f7ca535a64546f7a9b4421dad46c484`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:32.663484 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f6bed889-3620-4168-98f6-44df8c94ed04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:32.692546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac57cd-6865-4f65-b368-45de0b1fc4b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.694729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac57cd-6865-4f65-b368-45de0b1fc4b3 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.704039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1257/2507] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:32.710833 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:32.732785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1251/2508] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:32 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:32.744216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac57cd-6865-4f65-b368-45de0b1fc4b3 service nova] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:32.745657 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-32ac57cd-6865-4f65-b368-45de0b1fc4b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:32.802313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac57cd-6865-4f65-b368-45de0b1fc4b3 service nova] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:32.836341 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:32.852978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719068a-e39f-4c37-a55d-c61ec0c02d2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:32.853613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719068a-e39f-4c37-a55d-c61ec0c02d2a service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:32.853934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719068a-e39f-4c37-a55d-c61ec0c02d2a service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:32.854164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719068a-e39f-4c37-a55d-c61ec0c02d2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:32.863661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719068a-e39f-4c37-a55d-c61ec0c02d2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:32.865022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719068a-e39f-4c37-a55d-c61ec0c02d2a service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:32.875198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1258/2509] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:32 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:33.033769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bed889-3620-4168-98f6-44df8c94ed04 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:33.102596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bed889-3620-4168-98f6-44df8c94ed04 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:33.107081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:33.107735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:33.108016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:33.108286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] PATH_INFO: `/v3/users/aa2186ef8f8e4312b94c2dc57787eb93` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:33.109419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa2186ef8f8e4312b94c2dc57787eb93)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:33.111367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:33.159068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa2186ef8f8e4312b94c2dc57787eb93'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.159723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa2186ef8f8e4312b94c2dc57787eb93'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.166569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa2186ef8f8e4312b94c2dc57787eb93'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.173710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa2186ef8f8e4312b94c2dc57787eb93'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.174319 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa2186ef8f8e4312b94c2dc57787eb93'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.191753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1252/2510] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:33 2022] DELETE /identity/v3/users/aa2186ef8f8e4312b94c2dc57787eb93 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:33.196228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb84f9-8d71-4560-948f-986681b27a48 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:33.196517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb84f9-8d71-4560-948f-986681b27a48 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:33.196791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb84f9-8d71-4560-948f-986681b27a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:33.216288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-bdbb84f9-8d71-4560-948f-986681b27a48 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:33.217489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-bdbb84f9-8d71-4560-948f-986681b27a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:33.275223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1259/2511] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:33.285004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:33.286407 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:33.344175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d767e-eafc-4151-b2ef-631a0bacbfba admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:33.347748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5d320-ab5c-4dc6-9f87-219d4be59b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:33.348350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5d320-ab5c-4dc6-9f87-219d4be59b09 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:33.348630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5d320-ab5c-4dc6-9f87-219d4be59b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:33.348904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5d320-ab5c-4dc6-9f87-219d4be59b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:33.357851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5d320-ab5c-4dc6-9f87-219d4be59b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:33.359476 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5d320-ab5c-4dc6-9f87-219d4be59b09 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:33.370006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1253/2512] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:33.631011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb84f9-8d71-4560-948f-986681b27a48 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:33.632507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-bdbb84f9-8d71-4560-948f-986681b27a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:33.644675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb84f9-8d71-4560-948f-986681b27a48 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:33.649346 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:33.650105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:33.650444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:33.650775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] PATH_INFO: `/v3/projects/adc94ec120244542bac3f79a50b96c8a` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:33.652140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=adc94ec120244542bac3f79a50b96c8a)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:33.654623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:33.705273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adc94ec120244542bac3f79a50b96c8a', 'request_id': 'req-ee000e87-f330-4947-8ba9-0ad440ecfedc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.711733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adc94ec120244542bac3f79a50b96c8a', 'request_id': 'req-ee000e87-f330-4947-8ba9-0ad440ecfedc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.718122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adc94ec120244542bac3f79a50b96c8a', 'request_id': 'req-ee000e87-f330-4947-8ba9-0ad440ecfedc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.728216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] The token cache is being invalidate because project adc94ec120244542bac3f79a50b96c8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:33.728505 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.729126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.730623 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1260/2513] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:33 2022] DELETE /identity/v3/projects/adc94ec120244542bac3f79a50b96c8a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:33.735078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5d320-ab5c-4dc6-9f87-219d4be59b09 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:33.744510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5d320-ab5c-4dc6-9f87-219d4be59b09 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:33.748971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:33.801738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:33.802375 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:33.802634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:33.802964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] PATH_INFO: `/v3/users/2f7ca535a64546f7a9b4421dad46c484` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:33.804222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7ca535a64546f7a9b4421dad46c484)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:33.806224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:33.851340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7ca535a64546f7a9b4421dad46c484'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.851970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7ca535a64546f7a9b4421dad46c484'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.859143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7ca535a64546f7a9b4421dad46c484'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.865400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7ca535a64546f7a9b4421dad46c484'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.866020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-414e902e-bec5-4f0d-8d59-1ba41030dad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7ca535a64546f7a9b4421dad46c484'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:33.882936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1254/2514] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:33 2022] DELETE /identity/v3/users/2f7ca535a64546f7a9b4421dad46c484 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:34.153324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:34.156422 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:34.172329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee000e87-f330-4947-8ba9-0ad440ecfedc admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:34.233011 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:34.234230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:34.234512 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:34.234765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] PATH_INFO: `/v3/projects/a4ac0b9e1a9947948cbc2cbbe89b3ac2` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:34.235902 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ac0b9e1a9947948cbc2cbbe89b3ac2)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:34.237910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:34.244115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:34.295056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ac0b9e1a9947948cbc2cbbe89b3ac2', 'request_id': 'req-1b365839-d830-40c9-a304-fe24bf6d4683'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:34.303471 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ac0b9e1a9947948cbc2cbbe89b3ac2', 'request_id': 'req-1b365839-d830-40c9-a304-fe24bf6d4683'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:34.309998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ac0b9e1a9947948cbc2cbbe89b3ac2', 'request_id': 'req-1b365839-d830-40c9-a304-fe24bf6d4683'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:34.321169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] The token cache is being invalidate because project a4ac0b9e1a9947948cbc2cbbe89b3ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:34.321599 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:34.322639 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-1b365839-d830-40c9-a304-fe24bf6d4683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:34.324316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1261/2515] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:34 2022] DELETE /identity/v3/projects/a4ac0b9e1a9947948cbc2cbbe89b3ac2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:34.330750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64b975-16b6-41e5-aa21-8aaa7931c975 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:34.331248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64b975-16b6-41e5-aa21-8aaa7931c975 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:34.331343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64b975-16b6-41e5-aa21-8aaa7931c975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:34.480991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-1e64b975-16b6-41e5-aa21-8aaa7931c975 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:34.482239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1e64b975-16b6-41e5-aa21-8aaa7931c975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:34.540012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1255/2516] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:34.547819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdc397-d1a5-492a-91c0-5276a431c1f6 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:34.548107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdc397-d1a5-492a-91c0-5276a431c1f6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:34.548389 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdc397-d1a5-492a-91c0-5276a431c1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:34.567697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-80cdc397-d1a5-492a-91c0-5276a431c1f6 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:34.568898 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-80cdc397-d1a5-492a-91c0-5276a431c1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:34.576663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1262/2517] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:34.581460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64b975-16b6-41e5-aa21-8aaa7931c975 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:34.590861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64b975-16b6-41e5-aa21-8aaa7931c975 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:34.595960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a92bf-b543-4f45-9052-9669af25fc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:34.596540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a92bf-b543-4f45-9052-9669af25fc26 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:34.596811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a92bf-b543-4f45-9052-9669af25fc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:34.597075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a92bf-b543-4f45-9052-9669af25fc26 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:34.597747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a92bf-b543-4f45-9052-9669af25fc26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:34.598041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a92bf-b543-4f45-9052-9669af25fc26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:34.599608 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a92bf-b543-4f45-9052-9669af25fc26 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:34.610757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1256/2518] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:02:34.614931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdc397-d1a5-492a-91c0-5276a431c1f6 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:34.624198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdc397-d1a5-492a-91c0-5276a431c1f6 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:34.629105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f482e4a-8309-4e7d-912e-9ebc40eceb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:34.629754 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f482e4a-8309-4e7d-912e-9ebc40eceb47 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:34.630040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f482e4a-8309-4e7d-912e-9ebc40eceb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:34.630342 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f482e4a-8309-4e7d-912e-9ebc40eceb47 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:34.630997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f482e4a-8309-4e7d-912e-9ebc40eceb47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:34.632943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f482e4a-8309-4e7d-912e-9ebc40eceb47 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:34.661203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1263/2519] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:34 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:34.665471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a92bf-b543-4f45-9052-9669af25fc26 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:34.675288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a92bf-b543-4f45-9052-9669af25fc26 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:34.687080 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:34.733957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0a754-68a5-4daf-a48d-09c09857d29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:34.734556 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0a754-68a5-4daf-a48d-09c09857d29e admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:34.734834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0a754-68a5-4daf-a48d-09c09857d29e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:34.735105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0a754-68a5-4daf-a48d-09c09857d29e admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:34.735733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0a754-68a5-4daf-a48d-09c09857d29e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:34.737664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0a754-68a5-4daf-a48d-09c09857d29e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:34.758774 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1257/2520] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:34 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:34.762971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f482e4a-8309-4e7d-912e-9ebc40eceb47 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:34.773707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f482e4a-8309-4e7d-912e-9ebc40eceb47 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:34.778254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6cf61-8b01-4f83-be3a-593908deecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:34.778841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6cf61-8b01-4f83-be3a-593908deecd6 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:34.779118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6cf61-8b01-4f83-be3a-593908deecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:34.779388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6cf61-8b01-4f83-be3a-593908deecd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:34.780025 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6cf61-8b01-4f83-be3a-593908deecd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:34.781675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6cf61-8b01-4f83-be3a-593908deecd6 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:34.792722 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1264/2521] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:34.796481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0a754-68a5-4daf-a48d-09c09857d29e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:34.806326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0a754-68a5-4daf-a48d-09c09857d29e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:34.811038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e18e43-9547-478f-b81d-034d84e71dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:34.812111 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e18e43-9547-478f-b81d-034d84e71dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:34.812392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e18e43-9547-478f-b81d-034d84e71dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:34.812663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e18e43-9547-478f-b81d-034d84e71dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:34.813352 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e18e43-9547-478f-b81d-034d84e71dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:34.814990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e18e43-9547-478f-b81d-034d84e71dd7 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:34.824980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1258/2522] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:34.830538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6cf61-8b01-4f83-be3a-593908deecd6 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:34.840480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6cf61-8b01-4f83-be3a-593908deecd6 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:34.845117 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb97393-03d9-4de2-93d3-706cc7b0d965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:34.845809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb97393-03d9-4de2-93d3-706cc7b0d965 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:34.846089 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb97393-03d9-4de2-93d3-706cc7b0d965 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:34.846366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb97393-03d9-4de2-93d3-706cc7b0d965 admin admin] PATH_INFO: `/v3/projects/dda7ccb8d03b4c94b225380654bad19e/users/50fd008a382e4e91a283e3da8dfde9a3/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:34.860483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb97393-03d9-4de2-93d3-706cc7b0d965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda7ccb8d03b4c94b225380654bad19e, user_id=50fd008a382e4e91a283e3da8dfde9a3, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:34.863167 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb97393-03d9-4de2-93d3-706cc7b0d965 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:34.876463 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1265/2523] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:34 2022] PUT /identity/v3/projects/dda7ccb8d03b4c94b225380654bad19e/users/50fd008a382e4e91a283e3da8dfde9a3/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:34.886779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b1bb6-ca2b-49a4-a966-bca2deca2bf5 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:34.887063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b1bb6-ca2b-49a4-a966-bca2deca2bf5 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:34.887333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b1bb6-ca2b-49a4-a966-bca2deca2bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:34.914568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-6b1b1bb6-ca2b-49a4-a966-bca2deca2bf5 None None] MFA Rules not processed for user `50fd008a382e4e91a283e3da8dfde9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:34.915695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6b1b1bb6-ca2b-49a4-a966-bca2deca2bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:34.977252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1259/2524] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:34.981759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb97393-03d9-4de2-93d3-706cc7b0d965 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:35.041272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb97393-03d9-4de2-93d3-706cc7b0d965 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:35.046514 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ed19d-5ee8-43ff-9709-59ca51598060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:35.047112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ed19d-5ee8-43ff-9709-59ca51598060 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:35.047391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ed19d-5ee8-43ff-9709-59ca51598060 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:35.047690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ed19d-5ee8-43ff-9709-59ca51598060 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:35.048317 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ed19d-5ee8-43ff-9709-59ca51598060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:35.050259 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ed19d-5ee8-43ff-9709-59ca51598060 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:35.077221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1266/2525] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:34 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:35.081579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b1bb6-ca2b-49a4-a966-bca2deca2bf5 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:35.091826 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b1bb6-ca2b-49a4-a966-bca2deca2bf5 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:35.145457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aace02-3596-42b4-818d-a6b090b0f2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:35.146078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aace02-3596-42b4-818d-a6b090b0f2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:35.146355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aace02-3596-42b4-818d-a6b090b0f2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:35.146622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aace02-3596-42b4-818d-a6b090b0f2f2 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:35.147248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aace02-3596-42b4-818d-a6b090b0f2f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:35.149182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aace02-3596-42b4-818d-a6b090b0f2f2 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:35.170818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1260/2526] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:35 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:35.175225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ed19d-5ee8-43ff-9709-59ca51598060 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:35.185384 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ed19d-5ee8-43ff-9709-59ca51598060 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:35.190746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5262c-41f6-4a34-be0d-46423e682561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:35.191340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5262c-41f6-4a34-be0d-46423e682561 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:35.191625 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5262c-41f6-4a34-be0d-46423e682561 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:35.191903 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5262c-41f6-4a34-be0d-46423e682561 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:35.192552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5262c-41f6-4a34-be0d-46423e682561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:35.194198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5262c-41f6-4a34-be0d-46423e682561 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:35.205343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1267/2527] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:35.209275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aace02-3596-42b4-818d-a6b090b0f2f2 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:35.219035 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aace02-3596-42b4-818d-a6b090b0f2f2 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:35.223606 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d9336-3efb-46b3-b0a2-7508528c0fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:35.224245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d9336-3efb-46b3-b0a2-7508528c0fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:35.224524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d9336-3efb-46b3-b0a2-7508528c0fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:35.224795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d9336-3efb-46b3-b0a2-7508528c0fe8 admin admin] PATH_INFO: `/v3/projects/ab02ffc64f4d4a838da3e6961c27b385/users/555d863328054bcba88bc7f6c6248d69/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:35.236108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:35.240871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d9336-3efb-46b3-b0a2-7508528c0fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab02ffc64f4d4a838da3e6961c27b385, user_id=555d863328054bcba88bc7f6c6248d69, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:35.243799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d9336-3efb-46b3-b0a2-7508528c0fe8 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:35.256698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1261/2528] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:35 2022] PUT /identity/v3/projects/ab02ffc64f4d4a838da3e6961c27b385/users/555d863328054bcba88bc7f6c6248d69/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:35.260645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5262c-41f6-4a34-be0d-46423e682561 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:35.267060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:35.271287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5262c-41f6-4a34-be0d-46423e682561 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:35.329675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab5d07-0e27-4451-8d93-6ee27f0f645c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:35.330283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab5d07-0e27-4451-8d93-6ee27f0f645c admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:35.330566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab5d07-0e27-4451-8d93-6ee27f0f645c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:35.330844 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab5d07-0e27-4451-8d93-6ee27f0f645c admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:35.331520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab5d07-0e27-4451-8d93-6ee27f0f645c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:35.333112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab5d07-0e27-4451-8d93-6ee27f0f645c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:35.342609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1268/2529] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:35.346719 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d9336-3efb-46b3-b0a2-7508528c0fe8 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:35.356857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d9336-3efb-46b3-b0a2-7508528c0fe8 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:35.361976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaeb4b5-4bf7-40ba-b2fa-337ab823f05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:35.362608 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaeb4b5-4bf7-40ba-b2fa-337ab823f05a admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:35.362885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaeb4b5-4bf7-40ba-b2fa-337ab823f05a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:35.363152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaeb4b5-4bf7-40ba-b2fa-337ab823f05a admin admin] PATH_INFO: `/v3/projects/ab02ffc64f4d4a838da3e6961c27b385/users/555d863328054bcba88bc7f6c6248d69/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:35.365077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaeb4b5-4bf7-40ba-b2fa-337ab823f05a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab02ffc64f4d4a838da3e6961c27b385, user_id=555d863328054bcba88bc7f6c6248d69, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:35.367713 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaeb4b5-4bf7-40ba-b2fa-337ab823f05a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:35.378436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1262/2530] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:35 2022] PUT /identity/v3/projects/ab02ffc64f4d4a838da3e6961c27b385/users/555d863328054bcba88bc7f6c6248d69/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:35.389693 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e0580-4166-43b2-8321-ad316fdf4b5c None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:35.389987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e0580-4166-43b2-8321-ad316fdf4b5c None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:35.390267 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e0580-4166-43b2-8321-ad316fdf4b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:35.419973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f09e0580-4166-43b2-8321-ad316fdf4b5c None None] MFA Rules not processed for user `555d863328054bcba88bc7f6c6248d69`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:35.421840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f09e0580-4166-43b2-8321-ad316fdf4b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:35.489222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1269/2531] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:35.500077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaeb4b5-4bf7-40ba-b2fa-337ab823f05a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:35.501903 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7aaeb4b5-4bf7-40ba-b2fa-337ab823f05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:35.559606 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaeb4b5-4bf7-40ba-b2fa-337ab823f05a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:35.563184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687f53c-3cf8-4441-8e6f-0ce24f841c02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:35.563776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687f53c-3cf8-4441-8e6f-0ce24f841c02 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:35.564047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687f53c-3cf8-4441-8e6f-0ce24f841c02 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:35.564313 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687f53c-3cf8-4441-8e6f-0ce24f841c02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:35.572958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687f53c-3cf8-4441-8e6f-0ce24f841c02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:35.574415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687f53c-3cf8-4441-8e6f-0ce24f841c02 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:35.583807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1263/2532] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:35.628328 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e0580-4166-43b2-8321-ad316fdf4b5c None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:35.629722 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f09e0580-4166-43b2-8321-ad316fdf4b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:35.641326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e0580-4166-43b2-8321-ad316fdf4b5c None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:35.644876 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb59293-dfd4-4ba7-84a7-8bcb829b5000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:35.645463 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59293-dfd4-4ba7-84a7-8bcb829b5000 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:35.645771 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59293-dfd4-4ba7-84a7-8bcb829b5000 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:35.646041 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59293-dfd4-4ba7-84a7-8bcb829b5000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:35.705379 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb59293-dfd4-4ba7-84a7-8bcb829b5000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:35.706763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb59293-dfd4-4ba7-84a7-8bcb829b5000 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:35.715390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1270/2533] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:38.994421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687f53c-3cf8-4441-8e6f-0ce24f841c02 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:39.062224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687f53c-3cf8-4441-8e6f-0ce24f841c02 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:39.066610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:39.067229 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:39.067501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:39.067788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] PATH_INFO: `/v3/users/50fd008a382e4e91a283e3da8dfde9a3` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:39.068886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50fd008a382e4e91a283e3da8dfde9a3)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:39.070817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:39.122900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50fd008a382e4e91a283e3da8dfde9a3'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.123548 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50fd008a382e4e91a283e3da8dfde9a3'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.130504 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50fd008a382e4e91a283e3da8dfde9a3'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.138664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50fd008a382e4e91a283e3da8dfde9a3'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.139363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50fd008a382e4e91a283e3da8dfde9a3'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.153225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:39.161031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1264/2534] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:38 2022] DELETE /identity/v3/users/50fd008a382e4e91a283e3da8dfde9a3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:39.165566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3634a6a-5be7-42eb-9051-11329b649e0d None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:39.166074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3634a6a-5be7-42eb-9051-11329b649e0d None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:39.166164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3634a6a-5be7-42eb-9051-11329b649e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:39.185058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a3634a6a-5be7-42eb-9051-11329b649e0d None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:39.197615 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a3634a6a-5be7-42eb-9051-11329b649e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:39.205838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:39.245919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1271/2535] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:39.256171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:39.257871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:39.316106 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb5099-df79-4c23-97ca-c2bc6ad37b53 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:39.319741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218e906-1e43-43c2-a82a-fb3e3c4780c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:39.320341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218e906-1e43-43c2-a82a-fb3e3c4780c3 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:39.320620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218e906-1e43-43c2-a82a-fb3e3c4780c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:39.320887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218e906-1e43-43c2-a82a-fb3e3c4780c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:39.330537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218e906-1e43-43c2-a82a-fb3e3c4780c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:39.331960 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218e906-1e43-43c2-a82a-fb3e3c4780c3 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:39.341214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1265/2536] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:39.602867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3634a6a-5be7-42eb-9051-11329b649e0d None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:39.604289 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a3634a6a-5be7-42eb-9051-11329b649e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:39.617101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3634a6a-5be7-42eb-9051-11329b649e0d None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:39.621956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:39.622585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:39.622860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:39.623127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] PATH_INFO: `/v3/projects/dda7ccb8d03b4c94b225380654bad19e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:39.624849 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda7ccb8d03b4c94b225380654bad19e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:39.626774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:39.680035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda7ccb8d03b4c94b225380654bad19e', 'request_id': 'req-5f308130-4692-4c95-9e77-88eb0649f4fd'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.688863 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda7ccb8d03b4c94b225380654bad19e', 'request_id': 'req-5f308130-4692-4c95-9e77-88eb0649f4fd'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.695402 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda7ccb8d03b4c94b225380654bad19e', 'request_id': 'req-5f308130-4692-4c95-9e77-88eb0649f4fd'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.706646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] The token cache is being invalidate because project dda7ccb8d03b4c94b225380654bad19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:39.706981 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.707655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.709687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1272/2537] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:39 2022] DELETE /identity/v3/projects/dda7ccb8d03b4c94b225380654bad19e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:39.713368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218e906-1e43-43c2-a82a-fb3e3c4780c3 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:39.722861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218e906-1e43-43c2-a82a-fb3e3c4780c3 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:39.780687 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:39.781315 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:39.781635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:39.781908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] PATH_INFO: `/v3/users/555d863328054bcba88bc7f6c6248d69` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:39.783158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=555d863328054bcba88bc7f6c6248d69)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:39.785221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:39.840621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555d863328054bcba88bc7f6c6248d69'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.841263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555d863328054bcba88bc7f6c6248d69'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.848184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555d863328054bcba88bc7f6c6248d69'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.855522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555d863328054bcba88bc7f6c6248d69'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.856291 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-37d0c2d1-7648-488e-89ea-dfb781e58e0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555d863328054bcba88bc7f6c6248d69'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:39.873537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1266/2538] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:39 2022] DELETE /identity/v3/users/555d863328054bcba88bc7f6c6248d69 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:40.153851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:40.155447 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:40.174753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f308130-4692-4c95-9e77-88eb0649f4fd admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:40.235492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:40.236139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.236423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.236702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] PATH_INFO: `/v3/projects/ab02ffc64f4d4a838da3e6961c27b385` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.237872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab02ffc64f4d4a838da3e6961c27b385)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:40.239800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:40.301491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab02ffc64f4d4a838da3e6961c27b385', 'request_id': 'req-3b6811fe-0fa3-4010-b976-d514d255aa78'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:40.303238 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:40.311116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab02ffc64f4d4a838da3e6961c27b385', 'request_id': 'req-3b6811fe-0fa3-4010-b976-d514d255aa78'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:40.320511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab02ffc64f4d4a838da3e6961c27b385', 'request_id': 'req-3b6811fe-0fa3-4010-b976-d514d255aa78'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:40.332446 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] The token cache is being invalidate because project ab02ffc64f4d4a838da3e6961c27b385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:40.332733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:40.333395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3b6811fe-0fa3-4010-b976-d514d255aa78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:40.334946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1273/2539] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:40 2022] DELETE /identity/v3/projects/ab02ffc64f4d4a838da3e6961c27b385 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:40.340182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917e359-309e-47c4-82af-40b5ec5c360e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.340650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917e359-309e-47c4-82af-40b5ec5c360e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.340762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917e359-309e-47c4-82af-40b5ec5c360e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.361417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-9917e359-309e-47c4-82af-40b5ec5c360e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:40.362703 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9917e359-309e-47c4-82af-40b5ec5c360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:40.376248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:40.428737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1267/2540] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:40.436820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d92812-ebde-46b0-9ae6-793b5aa0f3cf None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.437104 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d92812-ebde-46b0-9ae6-793b5aa0f3cf None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.437375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d92812-ebde-46b0-9ae6-793b5aa0f3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.458892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-c1d92812-ebde-46b0-9ae6-793b5aa0f3cf None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:40.460219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c1d92812-ebde-46b0-9ae6-793b5aa0f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:40.469155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1274/2541] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:40.473813 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917e359-309e-47c4-82af-40b5ec5c360e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:40.484207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917e359-309e-47c4-82af-40b5ec5c360e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:40.489657 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeccbf9-2d4a-4b48-ac76-04465f8c4604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:40.490305 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeccbf9-2d4a-4b48-ac76-04465f8c4604 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.490582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeccbf9-2d4a-4b48-ac76-04465f8c4604 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.490849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeccbf9-2d4a-4b48-ac76-04465f8c4604 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.491506 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeccbf9-2d4a-4b48-ac76-04465f8c4604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:40.491801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeccbf9-2d4a-4b48-ac76-04465f8c4604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:40.493424 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeccbf9-2d4a-4b48-ac76-04465f8c4604 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:40.506183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1268/2542] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:02:40.510664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d92812-ebde-46b0-9ae6-793b5aa0f3cf None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:40.520837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d92812-ebde-46b0-9ae6-793b5aa0f3cf None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:40.525618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e5368-f07f-483e-aefd-52246ccf2cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:40.526225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e5368-f07f-483e-aefd-52246ccf2cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.526503 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e5368-f07f-483e-aefd-52246ccf2cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.526776 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e5368-f07f-483e-aefd-52246ccf2cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.527427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e5368-f07f-483e-aefd-52246ccf2cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:40.529403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e5368-f07f-483e-aefd-52246ccf2cb7 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:40.561261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1275/2543] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:40 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:40.562437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeccbf9-2d4a-4b48-ac76-04465f8c4604 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:40.572786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeccbf9-2d4a-4b48-ac76-04465f8c4604 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:40.637009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174e4d0-dd0b-4191-818c-c5ef7dcb460e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:40.637718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174e4d0-dd0b-4191-818c-c5ef7dcb460e admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.638000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174e4d0-dd0b-4191-818c-c5ef7dcb460e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.638272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174e4d0-dd0b-4191-818c-c5ef7dcb460e admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.638907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174e4d0-dd0b-4191-818c-c5ef7dcb460e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:40.641363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174e4d0-dd0b-4191-818c-c5ef7dcb460e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:40.663326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1269/2544] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:40 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:40.667541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e5368-f07f-483e-aefd-52246ccf2cb7 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:40.677893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e5368-f07f-483e-aefd-52246ccf2cb7 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:40.682734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eb565-d97c-4edd-85bb-1db0e875a421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:40.683287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eb565-d97c-4edd-85bb-1db0e875a421 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.683794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eb565-d97c-4edd-85bb-1db0e875a421 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.684071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eb565-d97c-4edd-85bb-1db0e875a421 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.684726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eb565-d97c-4edd-85bb-1db0e875a421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:40.686399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eb565-d97c-4edd-85bb-1db0e875a421 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:40.697036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1276/2545] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:40.701068 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174e4d0-dd0b-4191-818c-c5ef7dcb460e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:40.712543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174e4d0-dd0b-4191-818c-c5ef7dcb460e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:40.717327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4abd87-d95c-45ba-9307-27bf9a073e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:40.717973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abd87-d95c-45ba-9307-27bf9a073e60 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.718253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abd87-d95c-45ba-9307-27bf9a073e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.718523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abd87-d95c-45ba-9307-27bf9a073e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.719201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4abd87-d95c-45ba-9307-27bf9a073e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:40.720790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4abd87-d95c-45ba-9307-27bf9a073e60 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:40.730930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1270/2546] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:40.735135 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eb565-d97c-4edd-85bb-1db0e875a421 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:40.744999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eb565-d97c-4edd-85bb-1db0e875a421 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:40.750145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78378f-03a5-4319-9b1f-6eef37623c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:40.750783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78378f-03a5-4319-9b1f-6eef37623c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.751063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78378f-03a5-4319-9b1f-6eef37623c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.751335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78378f-03a5-4319-9b1f-6eef37623c24 admin admin] PATH_INFO: `/v3/projects/fa59d981ddf046bfb486707bc62c72d3/users/be22a9c40e6640cd8d2f120b04cb6181/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.766196 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78378f-03a5-4319-9b1f-6eef37623c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa59d981ddf046bfb486707bc62c72d3, user_id=be22a9c40e6640cd8d2f120b04cb6181, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:40.768842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78378f-03a5-4319-9b1f-6eef37623c24 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:40.781663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1277/2547] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:40 2022] PUT /identity/v3/projects/fa59d981ddf046bfb486707bc62c72d3/users/be22a9c40e6640cd8d2f120b04cb6181/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:40.791840 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164a5b6-a74f-4aea-9c83-be5036e1115c None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.792122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164a5b6-a74f-4aea-9c83-be5036e1115c None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.792419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164a5b6-a74f-4aea-9c83-be5036e1115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.819735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5164a5b6-a74f-4aea-9c83-be5036e1115c None None] MFA Rules not processed for user `be22a9c40e6640cd8d2f120b04cb6181`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:40.820941 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5164a5b6-a74f-4aea-9c83-be5036e1115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:40.895271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1271/2548] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:40.899814 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78378f-03a5-4319-9b1f-6eef37623c24 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:40.964645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78378f-03a5-4319-9b1f-6eef37623c24 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:40.970637 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458854e-b097-4b8d-b580-4ebeeba7ec20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:40.971383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458854e-b097-4b8d-b580-4ebeeba7ec20 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:40.971686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458854e-b097-4b8d-b580-4ebeeba7ec20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:40.972013 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458854e-b097-4b8d-b580-4ebeeba7ec20 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:40.972674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458854e-b097-4b8d-b580-4ebeeba7ec20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:40.974917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458854e-b097-4b8d-b580-4ebeeba7ec20 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:41.002956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1278/2549] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:40 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:41.006968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164a5b6-a74f-4aea-9c83-be5036e1115c None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:41.018202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164a5b6-a74f-4aea-9c83-be5036e1115c None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:41.029157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:41.075500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f7d45-2eb3-4d83-8d08-273aab7e17e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:41.076113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7d45-2eb3-4d83-8d08-273aab7e17e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:41.076392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7d45-2eb3-4d83-8d08-273aab7e17e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:41.076663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7d45-2eb3-4d83-8d08-273aab7e17e5 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:41.077295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f7d45-2eb3-4d83-8d08-273aab7e17e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:41.079504 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f7d45-2eb3-4d83-8d08-273aab7e17e5 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:41.102145 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1272/2550] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:41 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:41.105907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458854e-b097-4b8d-b580-4ebeeba7ec20 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:41.117473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458854e-b097-4b8d-b580-4ebeeba7ec20 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:41.122296 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc1ccf-c8d5-4bab-a733-ef7e7b5fa755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:41.122895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc1ccf-c8d5-4bab-a733-ef7e7b5fa755 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:41.123177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc1ccf-c8d5-4bab-a733-ef7e7b5fa755 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:41.123453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc1ccf-c8d5-4bab-a733-ef7e7b5fa755 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:41.124103 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc1ccf-c8d5-4bab-a733-ef7e7b5fa755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:41.126361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc1ccf-c8d5-4bab-a733-ef7e7b5fa755 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:41.136992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1279/2551] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:41.141367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f7d45-2eb3-4d83-8d08-273aab7e17e5 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:41.151811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f7d45-2eb3-4d83-8d08-273aab7e17e5 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:41.156444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d562fc-faa1-472a-a281-e73bbfa15c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:41.157101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d562fc-faa1-472a-a281-e73bbfa15c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:41.157381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d562fc-faa1-472a-a281-e73bbfa15c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:41.157688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d562fc-faa1-472a-a281-e73bbfa15c4a admin admin] PATH_INFO: `/v3/projects/dd587c20af2e48a6ae6120e49f1aca53/users/685dabdf542c4307865d317717f95f79/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:41.173517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d562fc-faa1-472a-a281-e73bbfa15c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd587c20af2e48a6ae6120e49f1aca53, user_id=685dabdf542c4307865d317717f95f79, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:41.176174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d562fc-faa1-472a-a281-e73bbfa15c4a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:41.191966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1273/2552] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:41 2022] PUT /identity/v3/projects/dd587c20af2e48a6ae6120e49f1aca53/users/685dabdf542c4307865d317717f95f79/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:41.196361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc1ccf-c8d5-4bab-a733-ef7e7b5fa755 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:41.209408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc1ccf-c8d5-4bab-a733-ef7e7b5fa755 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:41.259387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:41.273673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd2b47-b088-42d9-b72a-f0950173e9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:41.274285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd2b47-b088-42d9-b72a-f0950173e9be admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:41.274581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd2b47-b088-42d9-b72a-f0950173e9be admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:41.274840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd2b47-b088-42d9-b72a-f0950173e9be admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:41.275518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd2b47-b088-42d9-b72a-f0950173e9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:41.277128 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd2b47-b088-42d9-b72a-f0950173e9be admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:41.287499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1280/2553] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:41.292531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d562fc-faa1-472a-a281-e73bbfa15c4a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:41.305045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d562fc-faa1-472a-a281-e73bbfa15c4a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:41.310667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27502-e36d-42be-8d5d-73b1339f7bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:41.311576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27502-e36d-42be-8d5d-73b1339f7bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:41.312401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27502-e36d-42be-8d5d-73b1339f7bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:41.312820 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27502-e36d-42be-8d5d-73b1339f7bdf admin admin] PATH_INFO: `/v3/projects/dd587c20af2e48a6ae6120e49f1aca53/users/685dabdf542c4307865d317717f95f79/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:41.315456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe27502-e36d-42be-8d5d-73b1339f7bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd587c20af2e48a6ae6120e49f1aca53, user_id=685dabdf542c4307865d317717f95f79, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:41.318540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe27502-e36d-42be-8d5d-73b1339f7bdf admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:41.329617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1274/2554] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:41 2022] PUT /identity/v3/projects/dd587c20af2e48a6ae6120e49f1aca53/users/685dabdf542c4307865d317717f95f79/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:41.340830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088333e2-706a-4c07-9f28-9f3c6f391525 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:41.341144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088333e2-706a-4c07-9f28-9f3c6f391525 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:41.341419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088333e2-706a-4c07-9f28-9f3c6f391525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:41.371650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-088333e2-706a-4c07-9f28-9f3c6f391525 None None] MFA Rules not processed for user `685dabdf542c4307865d317717f95f79`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:41.372868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-088333e2-706a-4c07-9f28-9f3c6f391525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:41.443705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1281/2555] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:41.454628 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27502-e36d-42be-8d5d-73b1339f7bdf admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:41.456835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9fe27502-e36d-42be-8d5d-73b1339f7bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:41.516101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27502-e36d-42be-8d5d-73b1339f7bdf admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:41.519998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462c4be-7d26-437c-b359-2462e757adfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:41.520610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462c4be-7d26-437c-b359-2462e757adfe service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:41.521032 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462c4be-7d26-437c-b359-2462e757adfe service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:41.521334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462c4be-7d26-437c-b359-2462e757adfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:41.531609 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462c4be-7d26-437c-b359-2462e757adfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:41.533046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462c4be-7d26-437c-b359-2462e757adfe service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:41.543773 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1275/2556] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:41 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:41.591163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088333e2-706a-4c07-9f28-9f3c6f391525 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:41.592728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-088333e2-706a-4c07-9f28-9f3c6f391525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:41.605356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088333e2-706a-4c07-9f28-9f3c6f391525 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:41.609533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c645b24-2457-40f9-89de-d5019f2b0074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:41.610150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c645b24-2457-40f9-89de-d5019f2b0074 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:41.610428 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c645b24-2457-40f9-89de-d5019f2b0074 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:41.610699 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c645b24-2457-40f9-89de-d5019f2b0074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:41.675757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c645b24-2457-40f9-89de-d5019f2b0074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:41.677304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c645b24-2457-40f9-89de-d5019f2b0074 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:41.688799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1282/2557] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:42.239491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462c4be-7d26-437c-b359-2462e757adfe service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:42.239491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6462c4be-7d26-437c-b359-2462e757adfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:42.327643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462c4be-7d26-437c-b359-2462e757adfe service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:42.332957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:42.333704 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:42.333987 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:42.334264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] PATH_INFO: `/v3/users/9d36c4b97e6b4df780ee04bc02981a30` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:42.335555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d36c4b97e6b4df780ee04bc02981a30)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:42.337785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:42.352925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:42.393402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d36c4b97e6b4df780ee04bc02981a30'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:42.394094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d36c4b97e6b4df780ee04bc02981a30'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:42.400868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d36c4b97e6b4df780ee04bc02981a30'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:42.409905 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d36c4b97e6b4df780ee04bc02981a30'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:42.410878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d36c4b97e6b4df780ee04bc02981a30'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:42.429831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1276/2558] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:42 2022] DELETE /identity/v3/users/9d36c4b97e6b4df780ee04bc02981a30 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:42.434403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609262eb-12c7-4006-9221-c9164e59aeed None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:42.434919 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609262eb-12c7-4006-9221-c9164e59aeed None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:42.435024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609262eb-12c7-4006-9221-c9164e59aeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:42.454080 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-609262eb-12c7-4006-9221-c9164e59aeed None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:42.455510 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-609262eb-12c7-4006-9221-c9164e59aeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:42.518775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1283/2559] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:42.528934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:42.530299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:42.595006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d962c4b-916e-413f-91c9-b109c9a30e56 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:42.598747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19ae40-21c8-482d-adae-ccdbc040048f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:42.599343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19ae40-21c8-482d-adae-ccdbc040048f service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:42.599614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19ae40-21c8-482d-adae-ccdbc040048f service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:42.599877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19ae40-21c8-482d-adae-ccdbc040048f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:42.610063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19ae40-21c8-482d-adae-ccdbc040048f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:42.611499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19ae40-21c8-482d-adae-ccdbc040048f service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:42.622027 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1277/2560] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:42.932841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609262eb-12c7-4006-9221-c9164e59aeed None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:42.934475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-609262eb-12c7-4006-9221-c9164e59aeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:42.949031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609262eb-12c7-4006-9221-c9164e59aeed None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:42.958458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:42.960646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:42.961155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:42.961634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] PATH_INFO: `/v3/projects/8203ec601e074484b26de4380c9c9257` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:42.970495 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8203ec601e074484b26de4380c9c9257)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:42.974674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:42.982909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=100 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:43.038339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8203ec601e074484b26de4380c9c9257', 'request_id': 'req-44a40b30-251d-4153-be6e-24e1212dff6b'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:43.050709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8203ec601e074484b26de4380c9c9257', 'request_id': 'req-44a40b30-251d-4153-be6e-24e1212dff6b'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:43.058155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8203ec601e074484b26de4380c9c9257', 'request_id': 'req-44a40b30-251d-4153-be6e-24e1212dff6b'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:43.072918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] The token cache is being invalidate because project 8203ec601e074484b26de4380c9c9257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:43.073219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:43.073918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-44a40b30-251d-4153-be6e-24e1212dff6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:43.078755 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1284/2561] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:42 2022] DELETE /identity/v3/projects/8203ec601e074484b26de4380c9c9257 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:43.083952 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a3321-6c04-459d-87a5-c5d8aae70763 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.084376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a3321-6c04-459d-87a5-c5d8aae70763 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.084745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a3321-6c04-459d-87a5-c5d8aae70763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.111057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ad4a3321-6c04-459d-87a5-c5d8aae70763 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:43.112309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ad4a3321-6c04-459d-87a5-c5d8aae70763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:43.218701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1278/2562] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:43.229461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc070d-6598-448c-8c85-c2530c4f90c8 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.229773 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc070d-6598-448c-8c85-c2530c4f90c8 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.230263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc070d-6598-448c-8c85-c2530c4f90c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.266356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2ebc070d-6598-448c-8c85-c2530c4f90c8 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:43.269068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2ebc070d-6598-448c-8c85-c2530c4f90c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:43.279988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1285/2563] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:43.284915 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a3321-6c04-459d-87a5-c5d8aae70763 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:43.295338 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a3321-6c04-459d-87a5-c5d8aae70763 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:43.302592 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4a8fd-c207-43e5-a800-d445ecff716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:43.303272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4a8fd-c207-43e5-a800-d445ecff716b admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.303619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4a8fd-c207-43e5-a800-d445ecff716b admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.303954 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4a8fd-c207-43e5-a800-d445ecff716b admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.304800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4a8fd-c207-43e5-a800-d445ecff716b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:43.305209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4a8fd-c207-43e5-a800-d445ecff716b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:43.306948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4a8fd-c207-43e5-a800-d445ecff716b admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:43.325547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1279/2564] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:02:43.330576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc070d-6598-448c-8c85-c2530c4f90c8 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:43.341370 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc070d-6598-448c-8c85-c2530c4f90c8 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:43.350622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9d1a9-338b-4d80-ab87-b3b6f110e550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:43.351711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9d1a9-338b-4d80-ab87-b3b6f110e550 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.352589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9d1a9-338b-4d80-ab87-b3b6f110e550 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.353094 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9d1a9-338b-4d80-ab87-b3b6f110e550 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.354231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9d1a9-338b-4d80-ab87-b3b6f110e550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:43.359464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9d1a9-338b-4d80-ab87-b3b6f110e550 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:43.392245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1286/2565] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:43 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:43.396740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4a8fd-c207-43e5-a800-d445ecff716b admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:43.411117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4a8fd-c207-43e5-a800-d445ecff716b admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:43.429810 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:43.473714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b6ed-b6fa-4c0b-bcc4-a56fac9c69b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:43.474334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b6ed-b6fa-4c0b-bcc4-a56fac9c69b4 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.474677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b6ed-b6fa-4c0b-bcc4-a56fac9c69b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.475014 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b6ed-b6fa-4c0b-bcc4-a56fac9c69b4 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.475715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058b6ed-b6fa-4c0b-bcc4-a56fac9c69b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:43.477740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058b6ed-b6fa-4c0b-bcc4-a56fac9c69b4 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:43.506176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9d1a9-338b-4d80-ab87-b3b6f110e550 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:43.509680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1280/2566] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:43 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:43.524563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9d1a9-338b-4d80-ab87-b3b6f110e550 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:43.531758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1a68b-0b91-4286-8fbc-f5cad750721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:43.532458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1a68b-0b91-4286-8fbc-f5cad750721b admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.532822 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1a68b-0b91-4286-8fbc-f5cad750721b admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.533166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1a68b-0b91-4286-8fbc-f5cad750721b admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.534141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1a68b-0b91-4286-8fbc-f5cad750721b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:43.536344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1a68b-0b91-4286-8fbc-f5cad750721b admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:43.551523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1287/2567] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:43.554335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b6ed-b6fa-4c0b-bcc4-a56fac9c69b4 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:43.565016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b6ed-b6fa-4c0b-bcc4-a56fac9c69b4 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:43.570778 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97d793-622f-437c-abf4-9b0632090c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:43.571566 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97d793-622f-437c-abf4-9b0632090c75 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.572300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97d793-622f-437c-abf4-9b0632090c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.572666 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97d793-622f-437c-abf4-9b0632090c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.573694 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97d793-622f-437c-abf4-9b0632090c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:43.575506 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97d793-622f-437c-abf4-9b0632090c75 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:43.591335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1281/2568] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:43.595821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1a68b-0b91-4286-8fbc-f5cad750721b admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:43.606066 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1a68b-0b91-4286-8fbc-f5cad750721b admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:43.611909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3edc6c-6865-4590-9943-a4a256c61220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:43.612718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3edc6c-6865-4590-9943-a4a256c61220 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.613000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3edc6c-6865-4590-9943-a4a256c61220 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.613277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3edc6c-6865-4590-9943-a4a256c61220 admin admin] PATH_INFO: `/v3/projects/2f3ba499701c40c888c10c968395a74e/users/69c54b3136f8436296e2eb6a3dd65e5c/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.630838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3edc6c-6865-4590-9943-a4a256c61220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3ba499701c40c888c10c968395a74e, user_id=69c54b3136f8436296e2eb6a3dd65e5c, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:43.633701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3edc6c-6865-4590-9943-a4a256c61220 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:43.647709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1288/2569] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:43 2022] PUT /identity/v3/projects/2f3ba499701c40c888c10c968395a74e/users/69c54b3136f8436296e2eb6a3dd65e5c/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:43.658055 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c5436-b1f4-453c-9f30-91793b07d59a None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.658333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c5436-b1f4-453c-9f30-91793b07d59a None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.658652 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c5436-b1f4-453c-9f30-91793b07d59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.688003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-126c5436-b1f4-453c-9f30-91793b07d59a None None] MFA Rules not processed for user `69c54b3136f8436296e2eb6a3dd65e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:43.689201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-126c5436-b1f4-453c-9f30-91793b07d59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:43.758064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1282/2570] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:43.762759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3edc6c-6865-4590-9943-a4a256c61220 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:43.830934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3edc6c-6865-4590-9943-a4a256c61220 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:43.835727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db58f6a-1309-4a4f-8449-b94484854502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:43.836332 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db58f6a-1309-4a4f-8449-b94484854502 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.836617 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db58f6a-1309-4a4f-8449-b94484854502 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.836893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db58f6a-1309-4a4f-8449-b94484854502 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.837602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db58f6a-1309-4a4f-8449-b94484854502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:43.839515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db58f6a-1309-4a4f-8449-b94484854502 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:43.869537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1289/2571] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:43 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:43.871215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c5436-b1f4-453c-9f30-91793b07d59a None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:43.881909 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c5436-b1f4-453c-9f30-91793b07d59a None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:43.941770 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbddab0-255f-4889-9f79-68185d075268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:43.942708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbddab0-255f-4889-9f79-68185d075268 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:43.942833 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbddab0-255f-4889-9f79-68185d075268 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:43.943031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbddab0-255f-4889-9f79-68185d075268 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:43.943736 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbddab0-255f-4889-9f79-68185d075268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:43.945715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbddab0-255f-4889-9f79-68185d075268 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:43.975519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1283/2572] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:43 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:43.979593 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db58f6a-1309-4a4f-8449-b94484854502 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:43.993172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db58f6a-1309-4a4f-8449-b94484854502 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:43.998511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32307e4-d405-4609-8ced-3beb6513a2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:43.999859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32307e4-d405-4609-8ced-3beb6513a2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:44.000340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32307e4-d405-4609-8ced-3beb6513a2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:44.000840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32307e4-d405-4609-8ced-3beb6513a2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:44.001610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32307e4-d405-4609-8ced-3beb6513a2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:44.003387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32307e4-d405-4609-8ced-3beb6513a2a9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:44.017090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1290/2573] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:44.023925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbddab0-255f-4889-9f79-68185d075268 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:44.039605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbddab0-255f-4889-9f79-68185d075268 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:44.044497 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489cfb9-b8a9-4b39-acdf-9b7566010580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:44.045172 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489cfb9-b8a9-4b39-acdf-9b7566010580 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:44.045447 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489cfb9-b8a9-4b39-acdf-9b7566010580 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:44.045754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489cfb9-b8a9-4b39-acdf-9b7566010580 admin admin] PATH_INFO: `/v3/projects/5ac529e1d27c479aae41185f9b7e0b7f/users/81edd3ad8fba457c94677f4fd8ff2823/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:44.064393 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489cfb9-b8a9-4b39-acdf-9b7566010580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac529e1d27c479aae41185f9b7e0b7f, user_id=81edd3ad8fba457c94677f4fd8ff2823, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:44.067422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489cfb9-b8a9-4b39-acdf-9b7566010580 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:44.069944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:44.082223 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1284/2574] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:44 2022] PUT /identity/v3/projects/5ac529e1d27c479aae41185f9b7e0b7f/users/81edd3ad8fba457c94677f4fd8ff2823/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:44.086261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32307e4-d405-4609-8ced-3beb6513a2a9 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:44.091390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:44.097712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32307e4-d405-4609-8ced-3beb6513a2a9 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:44.155163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6469d-0c4e-4a37-9d59-4164fb0ad399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:44.155791 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6469d-0c4e-4a37-9d59-4164fb0ad399 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:44.156073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6469d-0c4e-4a37-9d59-4164fb0ad399 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:44.156347 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6469d-0c4e-4a37-9d59-4164fb0ad399 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:44.156989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6469d-0c4e-4a37-9d59-4164fb0ad399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:44.158624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6469d-0c4e-4a37-9d59-4164fb0ad399 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:44.168223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1291/2575] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:44.172874 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489cfb9-b8a9-4b39-acdf-9b7566010580 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:44.182948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489cfb9-b8a9-4b39-acdf-9b7566010580 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:44.188203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d8d1-317a-49e3-8495-9eab982f582d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:44.188844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d8d1-317a-49e3-8495-9eab982f582d admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:44.189122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d8d1-317a-49e3-8495-9eab982f582d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:44.189390 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d8d1-317a-49e3-8495-9eab982f582d admin admin] PATH_INFO: `/v3/projects/5ac529e1d27c479aae41185f9b7e0b7f/users/81edd3ad8fba457c94677f4fd8ff2823/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:44.191326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8d8d1-317a-49e3-8495-9eab982f582d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac529e1d27c479aae41185f9b7e0b7f, user_id=81edd3ad8fba457c94677f4fd8ff2823, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:44.193970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8d8d1-317a-49e3-8495-9eab982f582d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:44.203918 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1285/2576] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:44 2022] PUT /identity/v3/projects/5ac529e1d27c479aae41185f9b7e0b7f/users/81edd3ad8fba457c94677f4fd8ff2823/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:44.214126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794b040-3981-4214-960a-b33a80f04355 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:44.214469 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794b040-3981-4214-960a-b33a80f04355 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:44.214747 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794b040-3981-4214-960a-b33a80f04355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:44.244621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-4794b040-3981-4214-960a-b33a80f04355 None None] MFA Rules not processed for user `81edd3ad8fba457c94677f4fd8ff2823`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:44.245845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4794b040-3981-4214-960a-b33a80f04355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:44.319886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1292/2577] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:44.333472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d8d1-317a-49e3-8495-9eab982f582d admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:44.335997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5ea8d8d1-317a-49e3-8495-9eab982f582d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:44.392503 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d8d1-317a-49e3-8495-9eab982f582d admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:44.396118 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335afcde-11f5-49a9-a266-1f452a65b977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:44.396711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335afcde-11f5-49a9-a266-1f452a65b977 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:44.396989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335afcde-11f5-49a9-a266-1f452a65b977 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:44.397259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335afcde-11f5-49a9-a266-1f452a65b977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:44.406821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335afcde-11f5-49a9-a266-1f452a65b977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:44.408263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335afcde-11f5-49a9-a266-1f452a65b977 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:44.417732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1286/2578] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:44.839713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794b040-3981-4214-960a-b33a80f04355 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:44.905366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794b040-3981-4214-960a-b33a80f04355 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:44.914480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:44.915476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:44.915852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:44.916254 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] PATH_INFO: `/v3/users/69c54b3136f8436296e2eb6a3dd65e5c` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:44.917450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c54b3136f8436296e2eb6a3dd65e5c)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:44.919669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:44.947419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:44.966228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c54b3136f8436296e2eb6a3dd65e5c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:44.966870 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c54b3136f8436296e2eb6a3dd65e5c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:44.973695 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c54b3136f8436296e2eb6a3dd65e5c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:44.985843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c54b3136f8436296e2eb6a3dd65e5c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:44.993086 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c54b3136f8436296e2eb6a3dd65e5c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:44.998955 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1293/2579] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:44 2022] DELETE /identity/v3/users/69c54b3136f8436296e2eb6a3dd65e5c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:45.004040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3bc2f-f5de-49fe-bacf-73ddbb858d7f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:45.004498 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3bc2f-f5de-49fe-bacf-73ddbb858d7f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:45.004621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3bc2f-f5de-49fe-bacf-73ddbb858d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:45.023315 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-52c3bc2f-f5de-49fe-bacf-73ddbb858d7f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:45.024514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-52c3bc2f-f5de-49fe-bacf-73ddbb858d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:45.081553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1287/2580] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:45.091218 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:45.092712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:45.156726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ef90d-4163-42cf-b14c-0daacfeb22e9 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:45.160487 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6400cb9-d0d5-4381-b6a9-f06deb2372bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:45.161099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6400cb9-d0d5-4381-b6a9-f06deb2372bb service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:45.161377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6400cb9-d0d5-4381-b6a9-f06deb2372bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:45.161684 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6400cb9-d0d5-4381-b6a9-f06deb2372bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:45.170703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6400cb9-d0d5-4381-b6a9-f06deb2372bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:45.172626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6400cb9-d0d5-4381-b6a9-f06deb2372bb service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:45.182862 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1294/2581] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:45.272050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3bc2f-f5de-49fe-bacf-73ddbb858d7f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:45.273870 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-52c3bc2f-f5de-49fe-bacf-73ddbb858d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:45.285821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3bc2f-f5de-49fe-bacf-73ddbb858d7f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:45.290271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:45.290894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:45.291195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:45.291643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] PATH_INFO: `/v3/users/be22a9c40e6640cd8d2f120b04cb6181` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:45.292789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be22a9c40e6640cd8d2f120b04cb6181)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:45.294732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:45.344091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be22a9c40e6640cd8d2f120b04cb6181'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.344798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be22a9c40e6640cd8d2f120b04cb6181'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.351904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be22a9c40e6640cd8d2f120b04cb6181'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.362279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be22a9c40e6640cd8d2f120b04cb6181'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.362894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be22a9c40e6640cd8d2f120b04cb6181'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.375735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:45.385601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1288/2582] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:45 2022] DELETE /identity/v3/users/be22a9c40e6640cd8d2f120b04cb6181 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:45.388829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33320c74-8cd8-4164-b9cd-c5b5204e7bb1 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:45.389168 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33320c74-8cd8-4164-b9cd-c5b5204e7bb1 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:45.389449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33320c74-8cd8-4164-b9cd-c5b5204e7bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:45.409974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-33320c74-8cd8-4164-b9cd-c5b5204e7bb1 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:45.411200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-33320c74-8cd8-4164-b9cd-c5b5204e7bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:45.472777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1295/2583] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:45.485856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:45.487972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:45.514339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33320c74-8cd8-4164-b9cd-c5b5204e7bb1 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:45.516311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-33320c74-8cd8-4164-b9cd-c5b5204e7bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:45.529795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33320c74-8cd8-4164-b9cd-c5b5204e7bb1 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:45.534940 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:45.535587 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:45.535865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:45.536137 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] PATH_INFO: `/v3/projects/2f3ba499701c40c888c10c968395a74e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:45.537292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f3ba499701c40c888c10c968395a74e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:45.539252 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:45.573927 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c096-afa8-4a04-8ec7-a7e24b665541 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:45.577790 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc07cf-f369-4868-9046-a43ef95a2e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:45.578389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc07cf-f369-4868-9046-a43ef95a2e8a service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:45.578661 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc07cf-f369-4868-9046-a43ef95a2e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:45.578923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc07cf-f369-4868-9046-a43ef95a2e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:45.584231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:45.589806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc07cf-f369-4868-9046-a43ef95a2e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:45.591418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f3ba499701c40c888c10c968395a74e', 'request_id': 'req-29555fa6-825e-4ff5-a55d-76c5c69d52e4'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.591574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc07cf-f369-4868-9046-a43ef95a2e8a service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:45.599792 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f3ba499701c40c888c10c968395a74e', 'request_id': 'req-29555fa6-825e-4ff5-a55d-76c5c69d52e4'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.602986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1289/2584] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:02:45.606304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f3ba499701c40c888c10c968395a74e', 'request_id': 'req-29555fa6-825e-4ff5-a55d-76c5c69d52e4'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.618690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] The token cache is being invalidate because project 2f3ba499701c40c888c10c968395a74e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:45.619058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.619804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.621591 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1296/2585] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:45 2022] DELETE /identity/v3/projects/2f3ba499701c40c888c10c968395a74e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:45.625839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc07cf-f369-4868-9046-a43ef95a2e8a service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:45.635818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc07cf-f369-4868-9046-a43ef95a2e8a service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:45.694910 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:45.695534 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:45.695827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:45.696132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] PATH_INFO: `/v3/users/81edd3ad8fba457c94677f4fd8ff2823` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:45.697227 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81edd3ad8fba457c94677f4fd8ff2823)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:45.699175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:45.745451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81edd3ad8fba457c94677f4fd8ff2823'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.746323 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81edd3ad8fba457c94677f4fd8ff2823'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.754168 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81edd3ad8fba457c94677f4fd8ff2823'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.763850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81edd3ad8fba457c94677f4fd8ff2823'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.764485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81edd3ad8fba457c94677f4fd8ff2823'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:45.782613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1290/2586] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:45 2022] DELETE /identity/v3/users/81edd3ad8fba457c94677f4fd8ff2823 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:45.887930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:45.890050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:45.903880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29555fa6-825e-4ff5-a55d-76c5c69d52e4 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:45.993830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:45.995097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:45.995313 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:45.995408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] PATH_INFO: `/v3/projects/fa59d981ddf046bfb486707bc62c72d3` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:45.996640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa59d981ddf046bfb486707bc62c72d3)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:45.998712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.044268 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.045900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.069609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa59d981ddf046bfb486707bc62c72d3', 'request_id': 'req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.071133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:46.079047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa59d981ddf046bfb486707bc62c72d3', 'request_id': 'req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.090492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa59d981ddf046bfb486707bc62c72d3', 'request_id': 'req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.103924 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] The token cache is being invalidate because project fa59d981ddf046bfb486707bc62c72d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:46.104077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.104679 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.106220 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1297/2587] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:45 2022] DELETE /identity/v3/projects/fa59d981ddf046bfb486707bc62c72d3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:46.109073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.109073 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.130906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105099b8-64d2-429c-9fa1-d3432da34046 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.175427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059e2ea-7d34-4da7-ab1c-89abedf3aa4d admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.179994 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.180674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.180956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.181230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] PATH_INFO: `/v3/projects/5ac529e1d27c479aae41185f9b7e0b7f` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.182093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cefab-d923-4741-8f5a-b0b72071f0fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.182364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac529e1d27c479aae41185f9b7e0b7f)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.182696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cefab-d923-4741-8f5a-b0b72071f0fe service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.182974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cefab-d923-4741-8f5a-b0b72071f0fe service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.183241 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cefab-d923-4741-8f5a-b0b72071f0fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.184272 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.185143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ba9cefab-d923-4741-8f5a-b0b72071f0fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.185768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:46.199020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cefab-d923-4741-8f5a-b0b72071f0fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.200462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cefab-d923-4741-8f5a-b0b72071f0fe service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.211226 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1291/2588] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:46.212096 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cefab-d923-4741-8f5a-b0b72071f0fe service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.213542 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ba9cefab-d923-4741-8f5a-b0b72071f0fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.227009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cefab-d923-4741-8f5a-b0b72071f0fe service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.241368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac529e1d27c479aae41185f9b7e0b7f', 'request_id': 'req-ae248acd-383f-4175-bc9a-ed2876ed79e9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.252959 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac529e1d27c479aae41185f9b7e0b7f', 'request_id': 'req-ae248acd-383f-4175-bc9a-ed2876ed79e9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.261052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac529e1d27c479aae41185f9b7e0b7f', 'request_id': 'req-ae248acd-383f-4175-bc9a-ed2876ed79e9'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.284838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] The token cache is being invalidate because project 5ac529e1d27c479aae41185f9b7e0b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:46.285132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.285834 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-ae248acd-383f-4175-bc9a-ed2876ed79e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.287359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1298/2589] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:46 2022] DELETE /identity/v3/projects/5ac529e1d27c479aae41185f9b7e0b7f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:46.295100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94cb3f-ebfa-49af-8049-c36c31043107 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.295467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94cb3f-ebfa-49af-8049-c36c31043107 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.297024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94cb3f-ebfa-49af-8049-c36c31043107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.297174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.297174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.297454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.297759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] PATH_INFO: `/v3/users/685dabdf542c4307865d317717f95f79` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.299702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=685dabdf542c4307865d317717f95f79)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.301350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.326914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2e94cb3f-ebfa-49af-8049-c36c31043107 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:46.329826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2e94cb3f-ebfa-49af-8049-c36c31043107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.340851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1299/2590] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:46.349132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcff182-6831-411e-ae90-03e699d49948 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.349417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcff182-6831-411e-ae90-03e699d49948 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.349726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcff182-6831-411e-ae90-03e699d49948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.364034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '685dabdf542c4307865d317717f95f79'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.365161 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '685dabdf542c4307865d317717f95f79'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.370832 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-3dcff182-6831-411e-ae90-03e699d49948 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:46.371984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3dcff182-6831-411e-ae90-03e699d49948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.375452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '685dabdf542c4307865d317717f95f79'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.380911 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1300/2591] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:46.385133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcff182-6831-411e-ae90-03e699d49948 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.387405 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '685dabdf542c4307865d317717f95f79'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.388429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '685dabdf542c4307865d317717f95f79'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.394714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcff182-6831-411e-ae90-03e699d49948 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.399364 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c9903-ddb8-40e7-9f0c-633cdae03c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.400024 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c9903-ddb8-40e7-9f0c-633cdae03c11 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.400308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c9903-ddb8-40e7-9f0c-633cdae03c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.400576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c9903-ddb8-40e7-9f0c-633cdae03c11 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.401226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c9903-ddb8-40e7-9f0c-633cdae03c11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:46.401549 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c9903-ddb8-40e7-9f0c-633cdae03c11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.402953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:46.403145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c9903-ddb8-40e7-9f0c-633cdae03c11 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.414210 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1292/2592] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:46 2022] DELETE /identity/v3/users/685dabdf542c4307865d317717f95f79 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:46.416329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1301/2593] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:02:46.421914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.423688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.498769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765fe89-8c63-4842-ade1-0e8ffb9fb048 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.505705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c082b8-72d3-4528-bade-67d38699a4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.506328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c082b8-72d3-4528-bade-67d38699a4fc admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.506611 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c082b8-72d3-4528-bade-67d38699a4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.506883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c082b8-72d3-4528-bade-67d38699a4fc admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.507607 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c082b8-72d3-4528-bade-67d38699a4fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.509530 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c082b8-72d3-4528-bade-67d38699a4fc admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.543450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1293/2594] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:46 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:46.545302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c9903-ddb8-40e7-9f0c-633cdae03c11 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.556831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c9903-ddb8-40e7-9f0c-633cdae03c11 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.644483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d8723-9708-4b82-8e51-4220ba5cae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.645157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d8723-9708-4b82-8e51-4220ba5cae71 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.645481 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d8723-9708-4b82-8e51-4220ba5cae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.645762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d8723-9708-4b82-8e51-4220ba5cae71 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.646404 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d8723-9708-4b82-8e51-4220ba5cae71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.648329 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d8723-9708-4b82-8e51-4220ba5cae71 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.674809 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1302/2595] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:46 2022] POST /identity/v3/users => generated 295 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:46.676659 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c082b8-72d3-4528-bade-67d38699a4fc admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.688841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c082b8-72d3-4528-bade-67d38699a4fc admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.694196 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783c0c8-cbca-41f0-a82b-b09d91a49029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.694859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783c0c8-cbca-41f0-a82b-b09d91a49029 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.695171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783c0c8-cbca-41f0-a82b-b09d91a49029 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.695440 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783c0c8-cbca-41f0-a82b-b09d91a49029 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.696116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783c0c8-cbca-41f0-a82b-b09d91a49029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.697859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783c0c8-cbca-41f0-a82b-b09d91a49029 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.710875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d8723-9708-4b82-8e51-4220ba5cae71 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.712837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-957d8723-9708-4b82-8e51-4220ba5cae71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.728523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1294/2596] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:46.728732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d8723-9708-4b82-8e51-4220ba5cae71 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.729976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783c0c8-cbca-41f0-a82b-b09d91a49029 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.732420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.733063 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.733701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.733986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] PATH_INFO: `/v3/projects/dd587c20af2e48a6ae6120e49f1aca53` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.738632 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd587c20af2e48a6ae6120e49f1aca53)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.742517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783c0c8-cbca-41f0-a82b-b09d91a49029 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.744454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.748244 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba51396-a6f9-4c63-8535-57e396d068f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.749158 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba51396-a6f9-4c63-8535-57e396d068f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.750196 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba51396-a6f9-4c63-8535-57e396d068f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.750557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba51396-a6f9-4c63-8535-57e396d068f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.751379 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba51396-a6f9-4c63-8535-57e396d068f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.753262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba51396-a6f9-4c63-8535-57e396d068f0 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.766801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1295/2597] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:46.770988 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba51396-a6f9-4c63-8535-57e396d068f0 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.782794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba51396-a6f9-4c63-8535-57e396d068f0 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.788733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860581fd-4bd5-4119-a2f4-06c8918436cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.789476 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860581fd-4bd5-4119-a2f4-06c8918436cb admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.789861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860581fd-4bd5-4119-a2f4-06c8918436cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.790139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860581fd-4bd5-4119-a2f4-06c8918436cb admin admin] PATH_INFO: `/v3/projects/a574d23921d244328eca6af5c7a03777/users/808d37e4c10740a7889b0730f083f3ce/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.808821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:46.809239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860581fd-4bd5-4119-a2f4-06c8918436cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a574d23921d244328eca6af5c7a03777, user_id=808d37e4c10740a7889b0730f083f3ce, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.810690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd587c20af2e48a6ae6120e49f1aca53', 'request_id': 'req-24d6ef13-b8d0-4318-a378-1d48e09416a1'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.812748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860581fd-4bd5-4119-a2f4-06c8918436cb admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.824580 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd587c20af2e48a6ae6120e49f1aca53', 'request_id': 'req-24d6ef13-b8d0-4318-a378-1d48e09416a1'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.828837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1296/2598] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:46 2022] PUT /identity/v3/projects/a574d23921d244328eca6af5c7a03777/users/808d37e4c10740a7889b0730f083f3ce/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:46.838060 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd587c20af2e48a6ae6120e49f1aca53', 'request_id': 'req-24d6ef13-b8d0-4318-a378-1d48e09416a1'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.840186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79998465-092d-4da2-a058-410180214ce3 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.840550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79998465-092d-4da2-a058-410180214ce3 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.840894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79998465-092d-4da2-a058-410180214ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.846945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] The token cache is being invalidate because project dd587c20af2e48a6ae6120e49f1aca53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:46.847322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.848076 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-24d6ef13-b8d0-4318-a378-1d48e09416a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:46.849927 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1303/2599] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:46 2022] DELETE /identity/v3/projects/dd587c20af2e48a6ae6120e49f1aca53 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:46.854300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd25b09-fee6-41c6-b6ba-a44228796934 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.854808 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd25b09-fee6-41c6-b6ba-a44228796934 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.855235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd25b09-fee6-41c6-b6ba-a44228796934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.872797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-79998465-092d-4da2-a058-410180214ce3 None None] MFA Rules not processed for user `808d37e4c10740a7889b0730f083f3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:46.874063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-79998465-092d-4da2-a058-410180214ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.878621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-2bd25b09-fee6-41c6-b6ba-a44228796934 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:46.880132 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2bd25b09-fee6-41c6-b6ba-a44228796934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.952824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1304/2600] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:46.955420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349343e-cb4b-45cf-b3cc-4684cd1294f3 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.955813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349343e-cb4b-45cf-b3cc-4684cd1294f3 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.956194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349343e-cb4b-45cf-b3cc-4684cd1294f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.962982 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1297/2601] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:46.967289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79998465-092d-4da2-a058-410180214ce3 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:46.969538 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-79998465-092d-4da2-a058-410180214ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.979061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a349343e-cb4b-45cf-b3cc-4684cd1294f3 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:46.980411 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a349343e-cb4b-45cf-b3cc-4684cd1294f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:46.982725 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79998465-092d-4da2-a058-410180214ce3 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:46.988333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaaee0-efd0-4c90-8071-7517d9f4100b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:46.989093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaaee0-efd0-4c90-8071-7517d9f4100b admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:46.992660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaaee0-efd0-4c90-8071-7517d9f4100b admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:46.992795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1305/2602] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:46.992895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaaee0-efd0-4c90-8071-7517d9f4100b admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:46.992895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaaee0-efd0-4c90-8071-7517d9f4100b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:46.992895 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaaee0-efd0-4c90-8071-7517d9f4100b admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:46.993954 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349343e-cb4b-45cf-b3cc-4684cd1294f3 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.004035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349343e-cb4b-45cf-b3cc-4684cd1294f3 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.009163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30e2d4-becf-4ef3-9354-a20c2e27b23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.009931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30e2d4-becf-4ef3-9354-a20c2e27b23f admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.010321 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30e2d4-becf-4ef3-9354-a20c2e27b23f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.010887 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30e2d4-becf-4ef3-9354-a20c2e27b23f admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.011426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30e2d4-becf-4ef3-9354-a20c2e27b23f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:47.011956 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30e2d4-becf-4ef3-9354-a20c2e27b23f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.013985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30e2d4-becf-4ef3-9354-a20c2e27b23f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.028450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1298/2603] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:46 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:47.029764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaaee0-efd0-4c90-8071-7517d9f4100b admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.031067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1306/2604] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:02:47.035502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30e2d4-becf-4ef3-9354-a20c2e27b23f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.037097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:47.047200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30e2d4-becf-4ef3-9354-a20c2e27b23f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.104675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaaee0-efd0-4c90-8071-7517d9f4100b admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.110296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbe14c-2ead-4019-991e-992766af0658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.111031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbe14c-2ead-4019-991e-992766af0658 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.111423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbe14c-2ead-4019-991e-992766af0658 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.111765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbe14c-2ead-4019-991e-992766af0658 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.112505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbe14c-2ead-4019-991e-992766af0658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.114517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbe14c-2ead-4019-991e-992766af0658 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.115951 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cbe28-0847-4df6-9d92-3b55a93d2c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.116689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cbe28-0847-4df6-9d92-3b55a93d2c18 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.116973 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cbe28-0847-4df6-9d92-3b55a93d2c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.117338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cbe28-0847-4df6-9d92-3b55a93d2c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.118014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cbe28-0847-4df6-9d92-3b55a93d2c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.120115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cbe28-0847-4df6-9d92-3b55a93d2c18 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.138334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1299/2605] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:47 2022] POST /identity/v3/users => generated 301 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:47.142183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbe14c-2ead-4019-991e-992766af0658 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.153353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbe14c-2ead-4019-991e-992766af0658 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.158411 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099bec8-90fa-4a6b-b8c2-5afb4e91f752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.159031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099bec8-90fa-4a6b-b8c2-5afb4e91f752 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.159339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099bec8-90fa-4a6b-b8c2-5afb4e91f752 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.163089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099bec8-90fa-4a6b-b8c2-5afb4e91f752 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.163513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1307/2606] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:47 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:47.163617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099bec8-90fa-4a6b-b8c2-5afb4e91f752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.163617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099bec8-90fa-4a6b-b8c2-5afb4e91f752 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.164091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cbe28-0847-4df6-9d92-3b55a93d2c18 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.177300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1300/2607] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:47.182249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099bec8-90fa-4a6b-b8c2-5afb4e91f752 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.194389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099bec8-90fa-4a6b-b8c2-5afb4e91f752 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.224765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:47.252507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cbe28-0847-4df6-9d92-3b55a93d2c18 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.257910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6fa28-d4d2-4f3f-a75c-6d0d6166d3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.258675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6fa28-d4d2-4f3f-a75c-6d0d6166d3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.259303 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6fa28-d4d2-4f3f-a75c-6d0d6166d3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.259651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6fa28-d4d2-4f3f-a75c-6d0d6166d3c5 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.260396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6fa28-d4d2-4f3f-a75c-6d0d6166d3c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.262669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6fa28-d4d2-4f3f-a75c-6d0d6166d3c5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.268647 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d96a76-6fb8-44a7-b325-06447ee794f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.269569 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d96a76-6fb8-44a7-b325-06447ee794f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.269926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d96a76-6fb8-44a7-b325-06447ee794f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.270270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d96a76-6fb8-44a7-b325-06447ee794f3 admin admin] PATH_INFO: `/v3/projects/61412e10d853416ead63c75aebf12f5f/users/a129a4c58cd347a5a7a2ff780e5b2186/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.288259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d96a76-6fb8-44a7-b325-06447ee794f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61412e10d853416ead63c75aebf12f5f, user_id=a129a4c58cd347a5a7a2ff780e5b2186, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.297037 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d96a76-6fb8-44a7-b325-06447ee794f3 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.303318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1308/2608] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:47 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:47.308733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6fa28-d4d2-4f3f-a75c-6d0d6166d3c5 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.315374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1301/2609] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:47 2022] PUT /identity/v3/projects/61412e10d853416ead63c75aebf12f5f/users/a129a4c58cd347a5a7a2ff780e5b2186/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:47.319562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d96a76-6fb8-44a7-b325-06447ee794f3 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.323244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6fa28-d4d2-4f3f-a75c-6d0d6166d3c5 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.390212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d96a76-6fb8-44a7-b325-06447ee794f3 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.396887 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205f15a-c068-45d9-a989-c76c2bf06b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.397682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f15a-c068-45d9-a989-c76c2bf06b34 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.398065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f15a-c068-45d9-a989-c76c2bf06b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.398443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f15a-c068-45d9-a989-c76c2bf06b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.399235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205f15a-c068-45d9-a989-c76c2bf06b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.401419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205f15a-c068-45d9-a989-c76c2bf06b34 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.403800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02926c1c-f239-4980-9209-dd8f16cbabbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.404762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926c1c-f239-4980-9209-dd8f16cbabbd admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.405297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926c1c-f239-4980-9209-dd8f16cbabbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.405794 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926c1c-f239-4980-9209-dd8f16cbabbd admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.406607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02926c1c-f239-4980-9209-dd8f16cbabbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.407849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:47.409181 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02926c1c-f239-4980-9209-dd8f16cbabbd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.416653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1302/2610] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:47.419494 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205f15a-c068-45d9-a989-c76c2bf06b34 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.424372 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1309/2611] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:47.425348 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02926c1c-f239-4980-9209-dd8f16cbabbd admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.435517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205f15a-c068-45d9-a989-c76c2bf06b34 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.435698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02926c1c-f239-4980-9209-dd8f16cbabbd admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.441015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bfb29-0b18-4faf-aebf-96cf6d71534e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.441647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832165d2-e109-445d-8eb1-08dc8ecb96f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.441820 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bfb29-0b18-4faf-aebf-96cf6d71534e admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.442239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bfb29-0b18-4faf-aebf-96cf6d71534e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.442527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bfb29-0b18-4faf-aebf-96cf6d71534e admin admin] PATH_INFO: `/v3/projects/61412e10d853416ead63c75aebf12f5f/users/a129a4c58cd347a5a7a2ff780e5b2186/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.442697 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832165d2-e109-445d-8eb1-08dc8ecb96f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.443068 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832165d2-e109-445d-8eb1-08dc8ecb96f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.443935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832165d2-e109-445d-8eb1-08dc8ecb96f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.444261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832165d2-e109-445d-8eb1-08dc8ecb96f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.446001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832165d2-e109-445d-8eb1-08dc8ecb96f0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.446373 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bfb29-0b18-4faf-aebf-96cf6d71534e admin admin] RBAC: Authorizing `identity:create_grant(project_id=61412e10d853416ead63c75aebf12f5f, user_id=a129a4c58cd347a5a7a2ff780e5b2186, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.449081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bfb29-0b18-4faf-aebf-96cf6d71534e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.461566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1310/2612] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:47.462663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832165d2-e109-445d-8eb1-08dc8ecb96f0 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.466197 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1303/2613] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:47 2022] PUT /identity/v3/projects/61412e10d853416ead63c75aebf12f5f/users/a129a4c58cd347a5a7a2ff780e5b2186/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:47.472923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f00ba-9a29-4093-b023-c77efeac300f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.473401 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f00ba-9a29-4093-b023-c77efeac300f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.473809 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f00ba-9a29-4093-b023-c77efeac300f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.475883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832165d2-e109-445d-8eb1-08dc8ecb96f0 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.507448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-394f00ba-9a29-4093-b023-c77efeac300f None None] MFA Rules not processed for user `a129a4c58cd347a5a7a2ff780e5b2186`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:47.509411 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-394f00ba-9a29-4093-b023-c77efeac300f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:47.536607 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92302ee-4baa-49b1-bcd0-666ffac86c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.537285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92302ee-4baa-49b1-bcd0-666ffac86c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.537735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92302ee-4baa-49b1-bcd0-666ffac86c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.538019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92302ee-4baa-49b1-bcd0-666ffac86c87 admin admin] PATH_INFO: `/v3/projects/4de789c0bf3a45d2be5ddc9ce4ce76a4/users/55c756d9fdeb4b0f9eed9c839d263efd/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.554935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92302ee-4baa-49b1-bcd0-666ffac86c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de789c0bf3a45d2be5ddc9ce4ce76a4, user_id=55c756d9fdeb4b0f9eed9c839d263efd, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.557691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92302ee-4baa-49b1-bcd0-666ffac86c87 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.572035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1311/2614] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:47 2022] PUT /identity/v3/projects/4de789c0bf3a45d2be5ddc9ce4ce76a4/users/55c756d9fdeb4b0f9eed9c839d263efd/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:47.583571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd87b3e-4f40-4d15-8c39-230093047d79 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.583861 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd87b3e-4f40-4d15-8c39-230093047d79 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.584144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd87b3e-4f40-4d15-8c39-230093047d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.589271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1304/2615] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:47.600815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f00ba-9a29-4093-b023-c77efeac300f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.602382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-394f00ba-9a29-4093-b023-c77efeac300f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:47.622159 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-efd87b3e-4f40-4d15-8c39-230093047d79 None None] MFA Rules not processed for user `55c756d9fdeb4b0f9eed9c839d263efd`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:47.623393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-efd87b3e-4f40-4d15-8c39-230093047d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:47.657297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:47.663603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f00ba-9a29-4093-b023-c77efeac300f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.704074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1312/2616] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:47.708705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd87b3e-4f40-4d15-8c39-230093047d79 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.719139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be101-4ba6-46ec-80c5-078ac502e58d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.720139 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be101-4ba6-46ec-80c5-078ac502e58d service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.720443 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be101-4ba6-46ec-80c5-078ac502e58d service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.720718 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be101-4ba6-46ec-80c5-078ac502e58d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.776215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:47.778764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd87b3e-4f40-4d15-8c39-230093047d79 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.784586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925e7ed-a7e3-4d0c-916a-0a7b06ecb839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.785344 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e7ed-a7e3-4d0c-916a-0a7b06ecb839 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.785660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e7ed-a7e3-4d0c-916a-0a7b06ecb839 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.786072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e7ed-a7e3-4d0c-916a-0a7b06ecb839 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.787019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925e7ed-a7e3-4d0c-916a-0a7b06ecb839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.789082 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925e7ed-a7e3-4d0c-916a-0a7b06ecb839 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.801899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804be101-4ba6-46ec-80c5-078ac502e58d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.803454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804be101-4ba6-46ec-80c5-078ac502e58d service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.826479 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1313/2617] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:47 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:47.828053 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925e7ed-a7e3-4d0c-916a-0a7b06ecb839 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.830034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1305/2618] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:02:47.908194 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925e7ed-a7e3-4d0c-916a-0a7b06ecb839 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.913610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2afddb-fff6-4528-bb9e-41b250e06892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.914325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2afddb-fff6-4528-bb9e-41b250e06892 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.914711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2afddb-fff6-4528-bb9e-41b250e06892 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.915088 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2afddb-fff6-4528-bb9e-41b250e06892 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.915835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2afddb-fff6-4528-bb9e-41b250e06892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.921671 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2afddb-fff6-4528-bb9e-41b250e06892 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.955768 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1314/2619] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:47 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:47.960102 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be101-4ba6-46ec-80c5-078ac502e58d service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.979658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be101-4ba6-46ec-80c5-078ac502e58d service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:47.984530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2afddb-fff6-4528-bb9e-41b250e06892 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:47.987584 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c84d-d7a5-4a9d-bb12-7d883a6ac13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:47.988302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c84d-d7a5-4a9d-bb12-7d883a6ac13d admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:47.988650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c84d-d7a5-4a9d-bb12-7d883a6ac13d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:47.988990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c84d-d7a5-4a9d-bb12-7d883a6ac13d admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:47.989795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357c84d-d7a5-4a9d-bb12-7d883a6ac13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:47.991789 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357c84d-d7a5-4a9d-bb12-7d883a6ac13d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:47.999290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2afddb-fff6-4528-bb9e-41b250e06892 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:48.006252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1306/2620] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:48.006874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:48.007870 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c84d-d7a5-4a9d-bb12-7d883a6ac13d admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:48.008098 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:48.008635 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:48.010097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] PATH_INFO: `/v3/users/808d37e4c10740a7889b0730f083f3ce` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:48.012297 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=808d37e4c10740a7889b0730f083f3ce)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:48.016764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:48.022795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c84d-d7a5-4a9d-bb12-7d883a6ac13d admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:48.031192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69451ad-7d3a-4349-aca3-9f397d3d0bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:48.032289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69451ad-7d3a-4349-aca3-9f397d3d0bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:48.032289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69451ad-7d3a-4349-aca3-9f397d3d0bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:48.032687 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69451ad-7d3a-4349-aca3-9f397d3d0bcc admin admin] PATH_INFO: `/v3/projects/c809b517cb1f4d08a1b9898c85a3c1ea/users/6e0a39a5ef6e43f5a92318c1df4d1863/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:48.073583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69451ad-7d3a-4349-aca3-9f397d3d0bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c809b517cb1f4d08a1b9898c85a3c1ea, user_id=6e0a39a5ef6e43f5a92318c1df4d1863, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:48.073583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69451ad-7d3a-4349-aca3-9f397d3d0bcc admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:48.082731 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:48.095248 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '808d37e4c10740a7889b0730f083f3ce'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:48.095621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1307/2621] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:48 2022] PUT /identity/v3/projects/c809b517cb1f4d08a1b9898c85a3c1ea/users/6e0a39a5ef6e43f5a92318c1df4d1863/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:48.096683 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '808d37e4c10740a7889b0730f083f3ce'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:48.099872 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69451ad-7d3a-4349-aca3-9f397d3d0bcc admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:48.103383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d69451ad-7d3a-4349-aca3-9f397d3d0bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:48.110704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '808d37e4c10740a7889b0730f083f3ce'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:48.128200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '808d37e4c10740a7889b0730f083f3ce'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:48.137884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0a751a36-8213-41bb-bbf1-f35ee8af3087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '808d37e4c10740a7889b0730f083f3ce'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:48.151312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1315/2622] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:47 2022] DELETE /identity/v3/users/808d37e4c10740a7889b0730f083f3ce => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:48.154227 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593039e1-1c1b-417f-b7f0-8f997b20e858 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:48.154608 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593039e1-1c1b-417f-b7f0-8f997b20e858 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:48.154901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593039e1-1c1b-417f-b7f0-8f997b20e858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:48.183308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-593039e1-1c1b-417f-b7f0-8f997b20e858 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:48.184685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-593039e1-1c1b-417f-b7f0-8f997b20e858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:48.186372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69451ad-7d3a-4349-aca3-9f397d3d0bcc admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:48.253256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ff639-643c-45b3-b667-79d6152845a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:48.253900 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ff639-643c-45b3-b667-79d6152845a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:48.254184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ff639-643c-45b3-b667-79d6152845a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:48.254452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ff639-643c-45b3-b667-79d6152845a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:48.255100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ff639-643c-45b3-b667-79d6152845a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:48.256753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ff639-643c-45b3-b667-79d6152845a1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:48.257718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1316/2623] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:48.269845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593039e1-1c1b-417f-b7f0-8f997b20e858 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:48.271418 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-593039e1-1c1b-417f-b7f0-8f997b20e858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:48.271844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1308/2624] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:48.278134 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ff639-643c-45b3-b667-79d6152845a1 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:48.292829 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ff639-643c-45b3-b667-79d6152845a1 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:48.298287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64d02d-3a04-4d85-8fcc-40a6842f8e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:48.298823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64d02d-3a04-4d85-8fcc-40a6842f8e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:48.299099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64d02d-3a04-4d85-8fcc-40a6842f8e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:48.299456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64d02d-3a04-4d85-8fcc-40a6842f8e65 admin admin] PATH_INFO: `/v3/projects/c809b517cb1f4d08a1b9898c85a3c1ea/users/6e0a39a5ef6e43f5a92318c1df4d1863/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:48.301675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64d02d-3a04-4d85-8fcc-40a6842f8e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c809b517cb1f4d08a1b9898c85a3c1ea, user_id=6e0a39a5ef6e43f5a92318c1df4d1863, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:48.305387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64d02d-3a04-4d85-8fcc-40a6842f8e65 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:48.321931 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1309/2625] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:48 2022] PUT /identity/v3/projects/c809b517cb1f4d08a1b9898c85a3c1ea/users/6e0a39a5ef6e43f5a92318c1df4d1863/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:48.331962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6e20e-5eca-4407-a962-09eca6e66155 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:48.332277 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6e20e-5eca-4407-a962-09eca6e66155 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:48.332543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6e20e-5eca-4407-a962-09eca6e66155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:48.361808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-c9f6e20e-5eca-4407-a962-09eca6e66155 None None] MFA Rules not processed for user `6e0a39a5ef6e43f5a92318c1df4d1863`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:48.363432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c9f6e20e-5eca-4407-a962-09eca6e66155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:48.386516 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593039e1-1c1b-417f-b7f0-8f997b20e858 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:48.393176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:48.452036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1310/2626] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:48.459547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e4f01-16b4-47db-a119-7386adda18b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:48.460173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e4f01-16b4-47db-a119-7386adda18b7 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:48.460458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e4f01-16b4-47db-a119-7386adda18b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:48.460726 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e4f01-16b4-47db-a119-7386adda18b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:48.465785 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6e20e-5eca-4407-a962-09eca6e66155 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:48.467399 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c9f6e20e-5eca-4407-a962-09eca6e66155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:48.500202 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:48.543866 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e4f01-16b4-47db-a119-7386adda18b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:48.545327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e4f01-16b4-47db-a119-7386adda18b7 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:48.556766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1317/2627] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:48.558464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6e20e-5eca-4407-a962-09eca6e66155 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:48.562143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb5676-9065-407c-a08d-3242925ab8ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:48.562956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb5676-9065-407c-a08d-3242925ab8ef service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:48.563241 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb5676-9065-407c-a08d-3242925ab8ef service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:48.563511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb5676-9065-407c-a08d-3242925ab8ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:48.574664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb5676-9065-407c-a08d-3242925ab8ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:48.576471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb5676-9065-407c-a08d-3242925ab8ef service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:48.589603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1311/2628] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:02:48.925361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e4f01-16b4-47db-a119-7386adda18b7 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:48.927160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-494e4f01-16b4-47db-a119-7386adda18b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:48.946554 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e4f01-16b4-47db-a119-7386adda18b7 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:48.953391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:48.953391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:48.954163 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:48.954708 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] PATH_INFO: `/v3/projects/a574d23921d244328eca6af5c7a03777` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:48.956156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a574d23921d244328eca6af5c7a03777)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:48.958351 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:49.040971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a574d23921d244328eca6af5c7a03777', 'request_id': 'req-6c090488-c64c-4c9d-a8e7-e68afddc6edc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.052507 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a574d23921d244328eca6af5c7a03777', 'request_id': 'req-6c090488-c64c-4c9d-a8e7-e68afddc6edc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.061765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a574d23921d244328eca6af5c7a03777', 'request_id': 'req-6c090488-c64c-4c9d-a8e7-e68afddc6edc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.076596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] The token cache is being invalidate because project a574d23921d244328eca6af5c7a03777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:49.076901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.078943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.080723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1318/2629] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:48 2022] DELETE /identity/v3/projects/a574d23921d244328eca6af5c7a03777 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:49.084727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb5676-9065-407c-a08d-3242925ab8ef service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:49.087145 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:49.088577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:49.098575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb5676-9065-407c-a08d-3242925ab8ef service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:49.161219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090488-c64c-4c9d-a8e7-e68afddc6edc admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:49.164934 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:49.165679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:49.165964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:49.166242 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] PATH_INFO: `/v3/users/a129a4c58cd347a5a7a2ff780e5b2186` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:49.166650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:49.167592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:49.167738 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a129a4c58cd347a5a7a2ff780e5b2186)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:49.167839 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:49.167942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] PATH_INFO: `/v3/users/55c756d9fdeb4b0f9eed9c839d263efd` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:49.169059 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c756d9fdeb4b0f9eed9c839d263efd)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:49.169726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:49.171331 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:49.222198 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a129a4c58cd347a5a7a2ff780e5b2186'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.222880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a129a4c58cd347a5a7a2ff780e5b2186'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.224735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c756d9fdeb4b0f9eed9c839d263efd'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.225460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c756d9fdeb4b0f9eed9c839d263efd'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.227560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:49.231207 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a129a4c58cd347a5a7a2ff780e5b2186'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.240520 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c756d9fdeb4b0f9eed9c839d263efd'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.241224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a129a4c58cd347a5a7a2ff780e5b2186'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.242058 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7753a0ec-82e5-4f28-ae48-822f1568e58e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a129a4c58cd347a5a7a2ff780e5b2186'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.243966 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:49.256875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c756d9fdeb4b0f9eed9c839d263efd'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.265660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1312/2630] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:49 2022] DELETE /identity/v3/users/a129a4c58cd347a5a7a2ff780e5b2186 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:49.267355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c756d9fdeb4b0f9eed9c839d263efd'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:49.277711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1319/2631] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:49 2022] DELETE /identity/v3/users/55c756d9fdeb4b0f9eed9c839d263efd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:49.281049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ebb73-ff12-4b72-9367-c9a1a7d4400e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:49.281049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ebb73-ff12-4b72-9367-c9a1a7d4400e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:49.281049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ebb73-ff12-4b72-9367-c9a1a7d4400e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:49.311558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-638ebb73-ff12-4b72-9367-c9a1a7d4400e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:49.313472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-638ebb73-ff12-4b72-9367-c9a1a7d4400e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:49.404493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1313/2632] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:49.410974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:49.412585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:49.488388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcef0d8-4a54-4e42-8343-f9a96fd0d48e admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:49.493458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7df91-86c5-4a11-93a2-567985676b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:49.494123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7df91-86c5-4a11-93a2-567985676b29 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:49.494408 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7df91-86c5-4a11-93a2-567985676b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:49.494696 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7df91-86c5-4a11-93a2-567985676b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:49.508482 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7df91-86c5-4a11-93a2-567985676b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:49.510001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7df91-86c5-4a11-93a2-567985676b29 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:49.521640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1320/2633] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:02:49.891214 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ebb73-ff12-4b72-9367-c9a1a7d4400e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:49.892750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-638ebb73-ff12-4b72-9367-c9a1a7d4400e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:49.907815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ebb73-ff12-4b72-9367-c9a1a7d4400e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:49.908765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7df91-86c5-4a11-93a2-567985676b29 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:49.910704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ddf7df91-86c5-4a11-93a2-567985676b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:49.913594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:49.914349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:49.914730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:49.915116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] PATH_INFO: `/v3/projects/4de789c0bf3a45d2be5ddc9ce4ce76a4` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:49.916511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de789c0bf3a45d2be5ddc9ce4ce76a4)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:49.918854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:49.932680 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7df91-86c5-4a11-93a2-567985676b29 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:49.938867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:49.940173 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:49.940619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:49.941043 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] PATH_INFO: `/v3/projects/61412e10d853416ead63c75aebf12f5f` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:49.942412 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=61412e10d853416ead63c75aebf12f5f)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:49.945614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:50.000452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de789c0bf3a45d2be5ddc9ce4ce76a4', 'request_id': 'req-c3520d67-f177-4a13-8ca1-48e432c9176a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.004309 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61412e10d853416ead63c75aebf12f5f', 'request_id': 'req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.009485 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de789c0bf3a45d2be5ddc9ce4ce76a4', 'request_id': 'req-c3520d67-f177-4a13-8ca1-48e432c9176a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.014118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61412e10d853416ead63c75aebf12f5f', 'request_id': 'req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.017019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de789c0bf3a45d2be5ddc9ce4ce76a4', 'request_id': 'req-c3520d67-f177-4a13-8ca1-48e432c9176a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.022783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61412e10d853416ead63c75aebf12f5f', 'request_id': 'req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.028621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] The token cache is being invalidate because project 4de789c0bf3a45d2be5ddc9ce4ce76a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:50.029102 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.029808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.032288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1314/2634] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:49 2022] DELETE /identity/v3/projects/4de789c0bf3a45d2be5ddc9ce4ce76a4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:50.036069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:50.037487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.048493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] The token cache is being invalidate because project 61412e10d853416ead63c75aebf12f5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:50.049169 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.049864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-c47ab902-0506-4ed7-ab30-2e52df3b3e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.051715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1321/2635] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:49 2022] DELETE /identity/v3/projects/61412e10d853416ead63c75aebf12f5f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:50.058965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d13f9-5b52-4382-a583-26853c2579b4 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.059642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d13f9-5b52-4382-a583-26853c2579b4 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.059642 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d13f9-5b52-4382-a583-26853c2579b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.082028 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-5d9d13f9-5b52-4382-a583-26853c2579b4 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:50.083273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5d9d13f9-5b52-4382-a583-26853c2579b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.140594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:50.151090 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3520d67-f177-4a13-8ca1-48e432c9176a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:50.161596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:50.162082 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1322/2636] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:50.174472 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5a2de-a0a1-4859-8176-9500a617803e None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.174820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5a2de-a0a1-4859-8176-9500a617803e None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.175091 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5a2de-a0a1-4859-8176-9500a617803e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.216047 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d2e5a2de-a0a1-4859-8176-9500a617803e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:50.218118 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d2e5a2de-a0a1-4859-8176-9500a617803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.229187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1323/2637] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:50.229656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:50.230553 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.230925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.231295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] PATH_INFO: `/v3/users/6e0a39a5ef6e43f5a92318c1df4d1863` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.232869 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0a39a5ef6e43f5a92318c1df4d1863)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:50.233293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5a2de-a0a1-4859-8176-9500a617803e None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:50.235522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d2e5a2de-a0a1-4859-8176-9500a617803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.235642 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:50.251174 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5a2de-a0a1-4859-8176-9500a617803e None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:50.256040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:50.256668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.256950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.257276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] PATH_INFO: `/v3/users/9e5325f010ae4359bd6b514a1995614d` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.272209 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5325f010ae4359bd6b514a1995614d)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:50.274180 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:50.287651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0a39a5ef6e43f5a92318c1df4d1863'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.288430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0a39a5ef6e43f5a92318c1df4d1863'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.297857 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0a39a5ef6e43f5a92318c1df4d1863'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.306543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0a39a5ef6e43f5a92318c1df4d1863'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.307367 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0a39a5ef6e43f5a92318c1df4d1863'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.328189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1315/2638] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:50 2022] DELETE /identity/v3/users/6e0a39a5ef6e43f5a92318c1df4d1863 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:50.328568 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5325f010ae4359bd6b514a1995614d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.329072 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:50.329440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5325f010ae4359bd6b514a1995614d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.334630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.337141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5325f010ae4359bd6b514a1995614d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.350596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5325f010ae4359bd6b514a1995614d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.359845 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5325f010ae4359bd6b514a1995614d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.367375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1324/2639] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:50 2022] DELETE /identity/v3/users/9e5325f010ae4359bd6b514a1995614d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:50.414669 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818c0ad-1eda-4ad1-8266-99a2cfd5077e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:50.453292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:50.485684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9f92b-e64c-4001-8939-0726b67c04e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:50.486222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9f92b-e64c-4001-8939-0726b67c04e9 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.486500 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9f92b-e64c-4001-8939-0726b67c04e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.486771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9f92b-e64c-4001-8939-0726b67c04e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.487445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9f92b-e64c-4001-8939-0726b67c04e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:50.487747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9f92b-e64c-4001-8939-0726b67c04e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:50.489344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9f92b-e64c-4001-8939-0726b67c04e9 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:50.505144 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1316/2640] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:02:50.509820 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:50.513295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.541978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d5dfb-61c8-4cd7-9b0a-3063342a6bd2 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:50.548459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f1de3-f500-4768-8e48-f9a491b751af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:50.549206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f1de3-f500-4768-8e48-f9a491b751af admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.549596 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f1de3-f500-4768-8e48-f9a491b751af admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.550012 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f1de3-f500-4768-8e48-f9a491b751af admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.550723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f1de3-f500-4768-8e48-f9a491b751af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:50.552766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f1de3-f500-4768-8e48-f9a491b751af admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:50.563200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9f92b-e64c-4001-8939-0726b67c04e9 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:50.565342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3fa9f92b-e64c-4001-8939-0726b67c04e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.580741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9f92b-e64c-4001-8939-0726b67c04e9 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:50.588287 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:50.589160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.589865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.590292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] PATH_INFO: `/v3/projects/7f48b2441eec48c4ac784844e475a9e9` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.603121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1325/2641] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:50 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:50.604154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f1de3-f500-4768-8e48-f9a491b751af admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:50.605983 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f48b2441eec48c4ac784844e475a9e9)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:50.607890 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:50.667930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f48b2441eec48c4ac784844e475a9e9', 'request_id': 'req-1ebac7b0-9df0-4b85-b044-77cea35cf309'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.678069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f48b2441eec48c4ac784844e475a9e9', 'request_id': 'req-1ebac7b0-9df0-4b85-b044-77cea35cf309'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.679830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f1de3-f500-4768-8e48-f9a491b751af admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:50.685322 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f48b2441eec48c4ac784844e475a9e9', 'request_id': 'req-1ebac7b0-9df0-4b85-b044-77cea35cf309'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.697052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] The token cache is being invalidate because project 7f48b2441eec48c4ac784844e475a9e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:50.697339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.698065 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.699545 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1317/2642] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:50 2022] DELETE /identity/v3/projects/7f48b2441eec48c4ac784844e475a9e9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:50.700300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:50.701672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.771755 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebac7b0-9df0-4b85-b044-77cea35cf309 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:50.776538 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:50.777183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.777464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.777762 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] PATH_INFO: `/v3/projects/c809b517cb1f4d08a1b9898c85a3c1ea` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.778926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c809b517cb1f4d08a1b9898c85a3c1ea)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:50.781416 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:50.830297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:50.830745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:50.833126 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c809b517cb1f4d08a1b9898c85a3c1ea', 'request_id': 'req-2b737530-ce67-47b8-b276-d32c5ede8180'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.842060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c809b517cb1f4d08a1b9898c85a3c1ea', 'request_id': 'req-2b737530-ce67-47b8-b276-d32c5ede8180'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.850278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c809b517cb1f4d08a1b9898c85a3c1ea', 'request_id': 'req-2b737530-ce67-47b8-b276-d32c5ede8180'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.862341 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] The token cache is being invalidate because project c809b517cb1f4d08a1b9898c85a3c1ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:50.862624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.863245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-2b737530-ce67-47b8-b276-d32c5ede8180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:50.863517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea165a-cfb9-41df-8227-35e837f02a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:50.864125 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea165a-cfb9-41df-8227-35e837f02a80 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.864403 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea165a-cfb9-41df-8227-35e837f02a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.864674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea165a-cfb9-41df-8227-35e837f02a80 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.864872 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1318/2643] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:50 2022] DELETE /identity/v3/projects/c809b517cb1f4d08a1b9898c85a3c1ea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:50.865327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea165a-cfb9-41df-8227-35e837f02a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:50.866061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d5a2a-f806-4bee-996b-6aad1a144462 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.866343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d5a2a-f806-4bee-996b-6aad1a144462 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.866613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d5a2a-f806-4bee-996b-6aad1a144462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.867409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea165a-cfb9-41df-8227-35e837f02a80 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:50.887783 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-3c5d5a2a-f806-4bee-996b-6aad1a144462 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:50.891558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1326/2644] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:50 2022] POST /identity/v3/users => generated 289 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:50.891696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3c5d5a2a-f806-4bee-996b-6aad1a144462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.891798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cc543-ed80-4c3f-8894-59a3704991a0 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.891798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cc543-ed80-4c3f-8894-59a3704991a0 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.891798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cc543-ed80-4c3f-8894-59a3704991a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.910582 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-630cc543-ed80-4c3f-8894-59a3704991a0 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:50.911775 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-630cc543-ed80-4c3f-8894-59a3704991a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.949724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1319/2645] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:50.950811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d5a2a-f806-4bee-996b-6aad1a144462 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:50.952236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3c5d5a2a-f806-4bee-996b-6aad1a144462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:50.964089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d5a2a-f806-4bee-996b-6aad1a144462 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:50.970118 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acab0d-e982-4c74-b685-a16090b4c00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:50.970714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acab0d-e982-4c74-b685-a16090b4c00c admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.971010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acab0d-e982-4c74-b685-a16090b4c00c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.971265 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acab0d-e982-4c74-b685-a16090b4c00c admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.971907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acab0d-e982-4c74-b685-a16090b4c00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:50.973747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acab0d-e982-4c74-b685-a16090b4c00c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:50.978562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1327/2646] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:50.979957 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159555d-9cc4-47bc-8590-08b32f3b1896 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.980236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159555d-9cc4-47bc-8590-08b32f3b1896 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.980502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159555d-9cc4-47bc-8590-08b32f3b1896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.992673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1320/2647] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:50.992673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de76a-d05c-4e21-8d54-9678af2b33c0 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:50.992673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de76a-d05c-4e21-8d54-9678af2b33c0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:50.992673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de76a-d05c-4e21-8d54-9678af2b33c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:50.999287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e159555d-9cc4-47bc-8590-08b32f3b1896 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:51.000477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e159555d-9cc4-47bc-8590-08b32f3b1896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:51.008191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1328/2648] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:51.011958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159555d-9cc4-47bc-8590-08b32f3b1896 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.012075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-590de76a-d05c-4e21-8d54-9678af2b33c0 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:51.012075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-590de76a-d05c-4e21-8d54-9678af2b33c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:51.020420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1321/2649] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:51.021183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590de76a-d05c-4e21-8d54-9678af2b33c0 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.021838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159555d-9cc4-47bc-8590-08b32f3b1896 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.026508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8ef22-0770-4779-84be-3213753449b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.027092 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8ef22-0770-4779-84be-3213753449b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.027369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8ef22-0770-4779-84be-3213753449b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.027641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8ef22-0770-4779-84be-3213753449b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.028275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8ef22-0770-4779-84be-3213753449b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.029885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8ef22-0770-4779-84be-3213753449b5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.032877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590de76a-d05c-4e21-8d54-9678af2b33c0 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.037598 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cc524-82b4-495b-84f1-c132494c798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.038109 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cc524-82b4-495b-84f1-c132494c798a admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.038386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cc524-82b4-495b-84f1-c132494c798a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.038682 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cc524-82b4-495b-84f1-c132494c798a admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.039316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cc524-82b4-495b-84f1-c132494c798a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:51.039617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cc524-82b4-495b-84f1-c132494c798a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.040576 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1329/2650] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:51.041190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cc524-82b4-495b-84f1-c132494c798a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.041352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8ef22-0770-4779-84be-3213753449b5 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.050517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8ef22-0770-4779-84be-3213753449b5 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.054266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1322/2651] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:02:51.055150 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac921-bd4a-47d7-bd95-cb0078e40089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.055689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cc524-82b4-495b-84f1-c132494c798a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.055835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac921-bd4a-47d7-bd95-cb0078e40089 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.055980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac921-bd4a-47d7-bd95-cb0078e40089 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.056249 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac921-bd4a-47d7-bd95-cb0078e40089 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.056884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac921-bd4a-47d7-bd95-cb0078e40089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:51.057184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac921-bd4a-47d7-bd95-cb0078e40089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.058758 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac921-bd4a-47d7-bd95-cb0078e40089 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.066255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cc524-82b4-495b-84f1-c132494c798a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.070578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1330/2652] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:02:51.070920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a5bb8-4811-4568-95bf-7046f7fd59cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.071325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac921-bd4a-47d7-bd95-cb0078e40089 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.071549 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a5bb8-4811-4568-95bf-7046f7fd59cb admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.071821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a5bb8-4811-4568-95bf-7046f7fd59cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.072086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a5bb8-4811-4568-95bf-7046f7fd59cb admin admin] PATH_INFO: `/v3/projects/4b70f657e3d646edb71857f40926146d/users/bbd76f97aae64c2cad891a46951592af/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.083415 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac921-bd4a-47d7-bd95-cb0078e40089 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.088019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524791c5-b2c0-4ab9-9b18-03c7fea5aac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.088592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524791c5-b2c0-4ab9-9b18-03c7fea5aac5 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.088868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524791c5-b2c0-4ab9-9b18-03c7fea5aac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.089136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524791c5-b2c0-4ab9-9b18-03c7fea5aac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.089787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524791c5-b2c0-4ab9-9b18-03c7fea5aac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.089908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a5bb8-4811-4568-95bf-7046f7fd59cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b70f657e3d646edb71857f40926146d, user_id=bbd76f97aae64c2cad891a46951592af, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.091655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524791c5-b2c0-4ab9-9b18-03c7fea5aac5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.092716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a5bb8-4811-4568-95bf-7046f7fd59cb admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.108150 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1323/2653] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:51 2022] PUT /identity/v3/projects/4b70f657e3d646edb71857f40926146d/users/bbd76f97aae64c2cad891a46951592af/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:51.109368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a5bb8-4811-4568-95bf-7046f7fd59cb admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.123425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1331/2654] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:51 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:51.124588 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f68c79-db25-4b45-8795-ea05f42648b7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.124874 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f68c79-db25-4b45-8795-ea05f42648b7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.125761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f68c79-db25-4b45-8795-ea05f42648b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.152823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-e8f68c79-db25-4b45-8795-ea05f42648b7 None None] MFA Rules not processed for user `bbd76f97aae64c2cad891a46951592af`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:51.154081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-e8f68c79-db25-4b45-8795-ea05f42648b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:51.173930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a5bb8-4811-4568-95bf-7046f7fd59cb admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.200434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:51.214663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:51.224942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1332/2655] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:51.224942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f68c79-db25-4b45-8795-ea05f42648b7 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.236883 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc21e6-76d6-4005-9cb6-7d301c60d5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.237514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc21e6-76d6-4005-9cb6-7d301c60d5a3 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.237805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc21e6-76d6-4005-9cb6-7d301c60d5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.238078 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc21e6-76d6-4005-9cb6-7d301c60d5a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.238744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc21e6-76d6-4005-9cb6-7d301c60d5a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.240686 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc21e6-76d6-4005-9cb6-7d301c60d5a3 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.270978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1324/2656] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:51 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:51.274622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc21e6-76d6-4005-9cb6-7d301c60d5a3 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.276184 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a3bc21e6-76d6-4005-9cb6-7d301c60d5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:51.285687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f68c79-db25-4b45-8795-ea05f42648b7 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.334407 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc21e6-76d6-4005-9cb6-7d301c60d5a3 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.338076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b031a0f-e690-4a30-b591-5de28da6e684 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd04df68f9d84fc4bb981c8b69cc771a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.338793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031a0f-e690-4a30-b591-5de28da6e684 service glance] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.339070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031a0f-e690-4a30-b591-5de28da6e684 service glance] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.339335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031a0f-e690-4a30-b591-5de28da6e684 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.344284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbc06b-1add-485d-a458-06404ddf384f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.345377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbc06b-1add-485d-a458-06404ddf384f admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.345691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbc06b-1add-485d-a458-06404ddf384f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.345964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbc06b-1add-485d-a458-06404ddf384f admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.346608 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbc06b-1add-485d-a458-06404ddf384f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.348492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbc06b-1add-485d-a458-06404ddf384f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.370886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1333/2657] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:51 2022] POST /identity/v3/users => generated 302 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:51.371693 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbc06b-1add-485d-a458-06404ddf384f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.381987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbc06b-1add-485d-a458-06404ddf384f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.386695 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36fc4d-dfaa-4779-98ff-1e01cc7fecef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.387286 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36fc4d-dfaa-4779-98ff-1e01cc7fecef admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.387569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36fc4d-dfaa-4779-98ff-1e01cc7fecef admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.387846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36fc4d-dfaa-4779-98ff-1e01cc7fecef admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.388474 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36fc4d-dfaa-4779-98ff-1e01cc7fecef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.391146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36fc4d-dfaa-4779-98ff-1e01cc7fecef admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.400406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b031a0f-e690-4a30-b591-5de28da6e684 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.401904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b031a0f-e690-4a30-b591-5de28da6e684 service glance] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.411396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1325/2658] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:51.411861 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1334/2659] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:51 2022] POST /identity/v3/users => generated 283 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:51.412200 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b031a0f-e690-4a30-b591-5de28da6e684 service glance] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.415652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36fc4d-dfaa-4779-98ff-1e01cc7fecef admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.424380 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b031a0f-e690-4a30-b591-5de28da6e684 service glance] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.425157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36fc4d-dfaa-4779-98ff-1e01cc7fecef admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.429141 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0817f6f-12ec-43dc-ae0c-5d57238e3ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.429880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0817f6f-12ec-43dc-ae0c-5d57238e3ccb admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.430253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0817f6f-12ec-43dc-ae0c-5d57238e3ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.430368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbc014-0a16-4b80-8471-198fd3a04677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.430705 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0817f6f-12ec-43dc-ae0c-5d57238e3ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.430922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbc014-0a16-4b80-8471-198fd3a04677 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.431198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbc014-0a16-4b80-8471-198fd3a04677 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.431788 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0817f6f-12ec-43dc-ae0c-5d57238e3ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.433247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0817f6f-12ec-43dc-ae0c-5d57238e3ccb admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.433741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbc014-0a16-4b80-8471-198fd3a04677 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.434530 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbc014-0a16-4b80-8471-198fd3a04677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.436618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbc014-0a16-4b80-8471-198fd3a04677 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.447376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1326/2660] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:51.450765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1335/2661] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:51.450926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0817f6f-12ec-43dc-ae0c-5d57238e3ccb admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.451723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbc014-0a16-4b80-8471-198fd3a04677 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.462357 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbc014-0a16-4b80-8471-198fd3a04677 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.462603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0817f6f-12ec-43dc-ae0c-5d57238e3ccb admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.467057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012cea5-baa2-48d2-8259-52846cd39e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.467651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012cea5-baa2-48d2-8259-52846cd39e13 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.467923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012cea5-baa2-48d2-8259-52846cd39e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.468070 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a6b53-3258-4c00-927f-d46c66a713aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.468242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012cea5-baa2-48d2-8259-52846cd39e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.468884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012cea5-baa2-48d2-8259-52846cd39e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.470281 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a6b53-3258-4c00-927f-d46c66a713aa admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.470522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012cea5-baa2-48d2-8259-52846cd39e13 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.470634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a6b53-3258-4c00-927f-d46c66a713aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.470839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a6b53-3258-4c00-927f-d46c66a713aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.471497 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a6b53-3258-4c00-927f-d46c66a713aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.473262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a6b53-3258-4c00-927f-d46c66a713aa admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.481002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1336/2662] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:51.485465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012cea5-baa2-48d2-8259-52846cd39e13 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.492003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1327/2663] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:51.492868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a6b53-3258-4c00-927f-d46c66a713aa admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.496355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012cea5-baa2-48d2-8259-52846cd39e13 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.501793 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e7a59-135c-496b-9dba-24c81db53bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.502458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e7a59-135c-496b-9dba-24c81db53bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.502741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e7a59-135c-496b-9dba-24c81db53bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.503292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e7a59-135c-496b-9dba-24c81db53bc5 admin admin] PATH_INFO: `/v3/projects/47f39025c72e44f4a49e0aed7ef26073/users/ab8bb337acf046b5a3038131185d7427/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.505799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a6b53-3258-4c00-927f-d46c66a713aa admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.510596 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bae383-fb30-40a9-bb79-3b0496c4e98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.511248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bae383-fb30-40a9-bb79-3b0496c4e98f admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.511529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bae383-fb30-40a9-bb79-3b0496c4e98f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.511806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bae383-fb30-40a9-bb79-3b0496c4e98f admin admin] PATH_INFO: `/v3/projects/cf3c8c7eaf4d4d6489604ca886171a4e/users/e126910425e3470da153fa94baca88b5/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.521338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e7a59-135c-496b-9dba-24c81db53bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f39025c72e44f4a49e0aed7ef26073, user_id=ab8bb337acf046b5a3038131185d7427, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.524020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e7a59-135c-496b-9dba-24c81db53bc5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.534802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bae383-fb30-40a9-bb79-3b0496c4e98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf3c8c7eaf4d4d6489604ca886171a4e, user_id=e126910425e3470da153fa94baca88b5, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.537597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bae383-fb30-40a9-bb79-3b0496c4e98f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.539175 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1337/2664] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:51 2022] PUT /identity/v3/projects/47f39025c72e44f4a49e0aed7ef26073/users/ab8bb337acf046b5a3038131185d7427/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:51.552979 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b00a26-2299-49bf-a8c8-c3821be64ede None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.553424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b00a26-2299-49bf-a8c8-c3821be64ede None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.553853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b00a26-2299-49bf-a8c8-c3821be64ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.555330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1328/2665] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:51 2022] PUT /identity/v3/projects/cf3c8c7eaf4d4d6489604ca886171a4e/users/e126910425e3470da153fa94baca88b5/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:51.560741 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80779cd9-3463-4e47-8763-6f0e23018534 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.561105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80779cd9-3463-4e47-8763-6f0e23018534 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.561431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80779cd9-3463-4e47-8763-6f0e23018534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.583003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-80779cd9-3463-4e47-8763-6f0e23018534 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:51.584351 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-80779cd9-3463-4e47-8763-6f0e23018534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:51.589188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-15b00a26-2299-49bf-a8c8-c3821be64ede None None] MFA Rules not processed for user `ab8bb337acf046b5a3038131185d7427`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:51.590553 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-15b00a26-2299-49bf-a8c8-c3821be64ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:51.606673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:51.614757 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:51.657528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1329/2666] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:51.665365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1338/2667] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:51.669359 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80779cd9-3463-4e47-8763-6f0e23018534 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.674043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-80779cd9-3463-4e47-8763-6f0e23018534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:51.680880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b00a26-2299-49bf-a8c8-c3821be64ede None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:51.682857 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-15b00a26-2299-49bf-a8c8-c3821be64ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:51.751855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80779cd9-3463-4e47-8763-6f0e23018534 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.755834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b234f3c-8ca4-4ba5-bdd0-1a123f642e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.756458 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b234f3c-8ca4-4ba5-bdd0-1a123f642e75 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.756735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b234f3c-8ca4-4ba5-bdd0-1a123f642e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.757009 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b234f3c-8ca4-4ba5-bdd0-1a123f642e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.763585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b00a26-2299-49bf-a8c8-c3821be64ede None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:51.767248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b234f3c-8ca4-4ba5-bdd0-1a123f642e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.768972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b234f3c-8ca4-4ba5-bdd0-1a123f642e75 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.771134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b46f7-3efa-4f3d-a8f3-b99598198d3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd04df68f9d84fc4bb981c8b69cc771a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:51.771745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b46f7-3efa-4f3d-a8f3-b99598198d3c service glance] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:51.772031 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b46f7-3efa-4f3d-a8f3-b99598198d3c service glance] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:51.772298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b46f7-3efa-4f3d-a8f3-b99598198d3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:51.779650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1330/2668] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:02:51.783315 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b46f7-3efa-4f3d-a8f3-b99598198d3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:51.784762 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b46f7-3efa-4f3d-a8f3-b99598198d3c service glance] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:51.795391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1339/2669] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:52.574641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c580b1-218d-4d01-a2fa-79b10fe98cbf None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:52.575160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c580b1-218d-4d01-a2fa-79b10fe98cbf None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:52.575392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c580b1-218d-4d01-a2fa-79b10fe98cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:52.608891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-15c580b1-218d-4d01-a2fa-79b10fe98cbf None None] MFA Rules not processed for user `e126910425e3470da153fa94baca88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:52.609870 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-15c580b1-218d-4d01-a2fa-79b10fe98cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:52.693473 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1331/2670] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:52.698171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b46f7-3efa-4f3d-a8f3-b99598198d3c service glance] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:52.709748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b46f7-3efa-4f3d-a8f3-b99598198d3c service glance] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:52.714235 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1ac83-a381-4456-819b-6cd80d8835dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:52.714896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1ac83-a381-4456-819b-6cd80d8835dc admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:52.715244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1ac83-a381-4456-819b-6cd80d8835dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:52.715586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1ac83-a381-4456-819b-6cd80d8835dc admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:52.716323 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1ac83-a381-4456-819b-6cd80d8835dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:52.718362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1ac83-a381-4456-819b-6cd80d8835dc admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:52.751242 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1340/2671] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:52 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:52.755396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c580b1-218d-4d01-a2fa-79b10fe98cbf None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:52.767429 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c580b1-218d-4d01-a2fa-79b10fe98cbf None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:52.825780 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:52.834192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef2aac-84a4-48fe-8fda-8b0bfd1a8cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:52.834904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef2aac-84a4-48fe-8fda-8b0bfd1a8cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:52.835246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef2aac-84a4-48fe-8fda-8b0bfd1a8cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:52.835582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef2aac-84a4-48fe-8fda-8b0bfd1a8cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:52.836316 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef2aac-84a4-48fe-8fda-8b0bfd1a8cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:52.838340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef2aac-84a4-48fe-8fda-8b0bfd1a8cd7 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:52.861730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1332/2672] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:52 2022] POST /identity/v3/users => generated 309 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:52.865945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1ac83-a381-4456-819b-6cd80d8835dc admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:52.876850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1ac83-a381-4456-819b-6cd80d8835dc admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:52.881484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52257a-196b-4c1c-bff2-776644896c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:52.882119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52257a-196b-4c1c-bff2-776644896c3b admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:52.882396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52257a-196b-4c1c-bff2-776644896c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:52.882669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52257a-196b-4c1c-bff2-776644896c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:52.883319 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52257a-196b-4c1c-bff2-776644896c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:52.884920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52257a-196b-4c1c-bff2-776644896c3b admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:52.904295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1341/2673] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:52.906683 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef2aac-84a4-48fe-8fda-8b0bfd1a8cd7 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:52.922776 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef2aac-84a4-48fe-8fda-8b0bfd1a8cd7 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:52.930107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c7c9e-e5be-4d59-9623-34b7bea0c2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:52.930871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c7c9e-e5be-4d59-9623-34b7bea0c2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:52.931312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c7c9e-e5be-4d59-9623-34b7bea0c2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:52.931570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c7c9e-e5be-4d59-9623-34b7bea0c2a5 admin admin] PATH_INFO: `/v3/projects/d5801adf2f284cce88d6ebc374255710/users/da188b91bc0744d9b17218c6ced076a1/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:52.947827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c7c9e-e5be-4d59-9623-34b7bea0c2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5801adf2f284cce88d6ebc374255710, user_id=da188b91bc0744d9b17218c6ced076a1, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:52.950539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c7c9e-e5be-4d59-9623-34b7bea0c2a5 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:52.963613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1333/2674] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:52 2022] PUT /identity/v3/projects/d5801adf2f284cce88d6ebc374255710/users/da188b91bc0744d9b17218c6ced076a1/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:52.967610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52257a-196b-4c1c-bff2-776644896c3b admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:52.979388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52257a-196b-4c1c-bff2-776644896c3b admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:53.036203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb29c11-b0f0-4714-85de-ab08d21bdded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:53.036811 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb29c11-b0f0-4714-85de-ab08d21bdded admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:53.037084 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb29c11-b0f0-4714-85de-ab08d21bdded admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:53.037352 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb29c11-b0f0-4714-85de-ab08d21bdded admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:53.038019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb29c11-b0f0-4714-85de-ab08d21bdded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:53.039594 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb29c11-b0f0-4714-85de-ab08d21bdded admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:53.049676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1342/2675] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:53.054156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c7c9e-e5be-4d59-9623-34b7bea0c2a5 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:53.065034 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c7c9e-e5be-4d59-9623-34b7bea0c2a5 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:53.069701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f36f-1669-461a-96e1-8aaacf763348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:53.070347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f36f-1669-461a-96e1-8aaacf763348 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:53.070622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f36f-1669-461a-96e1-8aaacf763348 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:53.070891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f36f-1669-461a-96e1-8aaacf763348 admin admin] PATH_INFO: `/v3/projects/d5801adf2f284cce88d6ebc374255710/users/da188b91bc0744d9b17218c6ced076a1/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:53.072850 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3f36f-1669-461a-96e1-8aaacf763348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5801adf2f284cce88d6ebc374255710, user_id=da188b91bc0744d9b17218c6ced076a1, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:53.075489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3f36f-1669-461a-96e1-8aaacf763348 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:53.086178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1334/2676] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:53 2022] PUT /identity/v3/projects/d5801adf2f284cce88d6ebc374255710/users/da188b91bc0744d9b17218c6ced076a1/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:53.771250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2eded-05e3-4b2a-a9e8-c095342d1ccd None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:53.772419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2eded-05e3-4b2a-a9e8-c095342d1ccd None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:53.772419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2eded-05e3-4b2a-a9e8-c095342d1ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:53.803965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-fff2eded-05e3-4b2a-a9e8-c095342d1ccd None None] MFA Rules not processed for user `da188b91bc0744d9b17218c6ced076a1`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:53.805261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-fff2eded-05e3-4b2a-a9e8-c095342d1ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:53.821869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:53.884738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1343/2677] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:53 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:54.960547 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f36f-1669-461a-96e1-8aaacf763348 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:55.027130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f36f-1669-461a-96e1-8aaacf763348 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:55.031976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:55.032600 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:55.032876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:55.033145 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] PATH_INFO: `/v3/users/e126910425e3470da153fa94baca88b5` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:55.034326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e126910425e3470da153fa94baca88b5)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:55.036237 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:55.082740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e126910425e3470da153fa94baca88b5'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.083386 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e126910425e3470da153fa94baca88b5'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.090387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e126910425e3470da153fa94baca88b5'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.098881 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e126910425e3470da153fa94baca88b5'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.099400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e126910425e3470da153fa94baca88b5'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.119247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1335/2678] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:54 2022] DELETE /identity/v3/users/e126910425e3470da153fa94baca88b5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:55.309968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2eded-05e3-4b2a-a9e8-c095342d1ccd None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:55.320187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2eded-05e3-4b2a-a9e8-c095342d1ccd None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:55.377341 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:55.378000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:55.378278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:55.378551 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] PATH_INFO: `/v3/projects/cf3c8c7eaf4d4d6489604ca886171a4e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:55.379670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf3c8c7eaf4d4d6489604ca886171a4e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:55.381590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:55.428262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf3c8c7eaf4d4d6489604ca886171a4e', 'request_id': 'req-f3155431-bfcf-431b-b52f-c973b6e1fafc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.435785 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf3c8c7eaf4d4d6489604ca886171a4e', 'request_id': 'req-f3155431-bfcf-431b-b52f-c973b6e1fafc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.442020 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf3c8c7eaf4d4d6489604ca886171a4e', 'request_id': 'req-f3155431-bfcf-431b-b52f-c973b6e1fafc'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.452278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] The token cache is being invalidate because project cf3c8c7eaf4d4d6489604ca886171a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:55.453055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.453740 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.455459 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1344/2679] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:55 2022] DELETE /identity/v3/projects/cf3c8c7eaf4d4d6489604ca886171a4e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:55.459888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:55.461353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:55.475818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114ffa5-d8b1-4054-8929-0e8478228552 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:55.501687 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:55.536122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:55.536749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:55.537044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:55.537331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] PATH_INFO: `/v3/users/da188b91bc0744d9b17218c6ced076a1` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:55.538523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da188b91bc0744d9b17218c6ced076a1)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:55.540431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:55.601766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da188b91bc0744d9b17218c6ced076a1'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.601766 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da188b91bc0744d9b17218c6ced076a1'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.606827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da188b91bc0744d9b17218c6ced076a1'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.614649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da188b91bc0744d9b17218c6ced076a1'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.615266 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b918f77a-a03b-4dd0-8188-6424e8cf7979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da188b91bc0744d9b17218c6ced076a1'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.632409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1336/2680] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:55 2022] DELETE /identity/v3/users/da188b91bc0744d9b17218c6ced076a1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:55.806234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:55.807689 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:55.822255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3155431-bfcf-431b-b52f-c973b6e1fafc admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:55.866430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:55.892659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:55.893294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:55.893614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:55.893882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] PATH_INFO: `/v3/projects/d5801adf2f284cce88d6ebc374255710` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:55.894989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5801adf2f284cce88d6ebc374255710)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:55.896885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:55.941931 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5801adf2f284cce88d6ebc374255710', 'request_id': 'req-e75813b3-c97e-424b-b731-54476a8c6495'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.948841 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5801adf2f284cce88d6ebc374255710', 'request_id': 'req-e75813b3-c97e-424b-b731-54476a8c6495'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.955805 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5801adf2f284cce88d6ebc374255710', 'request_id': 'req-e75813b3-c97e-424b-b731-54476a8c6495'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.968460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] The token cache is being invalidate because project d5801adf2f284cce88d6ebc374255710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:55.968748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.969409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-e75813b3-c97e-424b-b731-54476a8c6495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:55.970921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1345/2681] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:55 2022] DELETE /identity/v3/projects/d5801adf2f284cce88d6ebc374255710 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:55.975906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924c1ab-72f1-4417-bb04-b13bbc420b4f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:55.976431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924c1ab-72f1-4417-bb04-b13bbc420b4f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:55.976431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924c1ab-72f1-4417-bb04-b13bbc420b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:55.995408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-1924c1ab-72f1-4417-bb04-b13bbc420b4f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:55.996585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1924c1ab-72f1-4417-bb04-b13bbc420b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:56.059510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1337/2682] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:56.067769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140502c9-c738-41e4-8185-b18bb854350a None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.068097 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140502c9-c738-41e4-8185-b18bb854350a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.068353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140502c9-c738-41e4-8185-b18bb854350a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.088782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-140502c9-c738-41e4-8185-b18bb854350a None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:56.090042 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-140502c9-c738-41e4-8185-b18bb854350a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:56.098087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1346/2683] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:56.102267 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924c1ab-72f1-4417-bb04-b13bbc420b4f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.112646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924c1ab-72f1-4417-bb04-b13bbc420b4f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.117425 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69f0d9-94dc-4df3-bc1a-e410914031f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.118175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69f0d9-94dc-4df3-bc1a-e410914031f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.118467 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69f0d9-94dc-4df3-bc1a-e410914031f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.118723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69f0d9-94dc-4df3-bc1a-e410914031f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.119396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69f0d9-94dc-4df3-bc1a-e410914031f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:56.119693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69f0d9-94dc-4df3-bc1a-e410914031f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.121400 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69f0d9-94dc-4df3-bc1a-e410914031f5 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.136770 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1338/2684] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:02:56.141149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140502c9-c738-41e4-8185-b18bb854350a None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.151363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140502c9-c738-41e4-8185-b18bb854350a None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.156325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9f87e-4fe9-4de9-9f50-291b4edf37ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.156946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9f87e-4fe9-4de9-9f50-291b4edf37ff admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.157263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9f87e-4fe9-4de9-9f50-291b4edf37ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.157597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9f87e-4fe9-4de9-9f50-291b4edf37ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.158434 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9f87e-4fe9-4de9-9f50-291b4edf37ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.160133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9f87e-4fe9-4de9-9f50-291b4edf37ff admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.188326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1347/2685] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:56 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:56.192353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69f0d9-94dc-4df3-bc1a-e410914031f5 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.202358 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69f0d9-94dc-4df3-bc1a-e410914031f5 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.232511 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:56.262924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59f686-9291-4115-b684-ad1b61099236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.263527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59f686-9291-4115-b684-ad1b61099236 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.263806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59f686-9291-4115-b684-ad1b61099236 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.264076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59f686-9291-4115-b684-ad1b61099236 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.264699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59f686-9291-4115-b684-ad1b61099236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.266673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59f686-9291-4115-b684-ad1b61099236 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.287110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1339/2686] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:56 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:56.293849 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9f87e-4fe9-4de9-9f50-291b4edf37ff admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.306077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9f87e-4fe9-4de9-9f50-291b4edf37ff admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.309714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d56153-4404-42f4-a857-533df58d4b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.310271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56153-4404-42f4-a857-533df58d4b56 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.310826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56153-4404-42f4-a857-533df58d4b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.310826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56153-4404-42f4-a857-533df58d4b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.311453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d56153-4404-42f4-a857-533df58d4b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.313657 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d56153-4404-42f4-a857-533df58d4b56 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.324497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1348/2687] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:56.329089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59f686-9291-4115-b684-ad1b61099236 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.343025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59f686-9291-4115-b684-ad1b61099236 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.348528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9228b4d-1110-42ef-9968-555790977792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.349178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9228b4d-1110-42ef-9968-555790977792 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.349469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9228b4d-1110-42ef-9968-555790977792 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.349743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9228b4d-1110-42ef-9968-555790977792 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.350426 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9228b4d-1110-42ef-9968-555790977792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.352061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9228b4d-1110-42ef-9968-555790977792 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.362739 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1340/2688] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:56.366961 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d56153-4404-42f4-a857-533df58d4b56 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.376947 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d56153-4404-42f4-a857-533df58d4b56 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.381400 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f361c-9d7c-4868-af5b-500294a673dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.382069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f361c-9d7c-4868-af5b-500294a673dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.382343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f361c-9d7c-4868-af5b-500294a673dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.382610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f361c-9d7c-4868-af5b-500294a673dc admin admin] PATH_INFO: `/v3/projects/82c536ee58aa410cbd0ef1b6dfe63d7c/users/ced98667e3a44b62adea9f4b95667680/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.397760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f361c-9d7c-4868-af5b-500294a673dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c536ee58aa410cbd0ef1b6dfe63d7c, user_id=ced98667e3a44b62adea9f4b95667680, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.400406 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f361c-9d7c-4868-af5b-500294a673dc admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.414023 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1349/2689] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:56 2022] PUT /identity/v3/projects/82c536ee58aa410cbd0ef1b6dfe63d7c/users/ced98667e3a44b62adea9f4b95667680/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:56.425387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad6630-55ff-45b5-b5eb-f7c83c66a7ed None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.425709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad6630-55ff-45b5-b5eb-f7c83c66a7ed None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.425977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad6630-55ff-45b5-b5eb-f7c83c66a7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.456491 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-bbad6630-55ff-45b5-b5eb-f7c83c66a7ed None None] MFA Rules not processed for user `ced98667e3a44b62adea9f4b95667680`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:56.457722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bbad6630-55ff-45b5-b5eb-f7c83c66a7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:56.524860 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1341/2690] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:56.529299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f361c-9d7c-4868-af5b-500294a673dc admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.592483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f361c-9d7c-4868-af5b-500294a673dc admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.597072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0bbab-7d0d-4ce4-9d46-e956aa8842c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.597711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0bbab-7d0d-4ce4-9d46-e956aa8842c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.597990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0bbab-7d0d-4ce4-9d46-e956aa8842c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.598258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0bbab-7d0d-4ce4-9d46-e956aa8842c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.598900 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0bbab-7d0d-4ce4-9d46-e956aa8842c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.600788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0bbab-7d0d-4ce4-9d46-e956aa8842c9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.615046 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:56.634815 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1350/2691] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:56 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:56.635779 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad6630-55ff-45b5-b5eb-f7c83c66a7ed None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.646969 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad6630-55ff-45b5-b5eb-f7c83c66a7ed None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.706753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc0349-340b-4940-bbc8-fc424ff2d285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.707343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc0349-340b-4940-bbc8-fc424ff2d285 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.707617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc0349-340b-4940-bbc8-fc424ff2d285 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.707884 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc0349-340b-4940-bbc8-fc424ff2d285 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.708527 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc0349-340b-4940-bbc8-fc424ff2d285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.710479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc0349-340b-4940-bbc8-fc424ff2d285 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.732385 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1342/2692] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:56 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:56.736440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0bbab-7d0d-4ce4-9d46-e956aa8842c9 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.746652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0bbab-7d0d-4ce4-9d46-e956aa8842c9 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.751698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604db7c6-16ff-45b5-a9fb-b230b5241de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.752292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604db7c6-16ff-45b5-a9fb-b230b5241de8 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.752571 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604db7c6-16ff-45b5-a9fb-b230b5241de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.752842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604db7c6-16ff-45b5-a9fb-b230b5241de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.753483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604db7c6-16ff-45b5-a9fb-b230b5241de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.755090 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604db7c6-16ff-45b5-a9fb-b230b5241de8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.765895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1351/2693] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:56.769688 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc0349-340b-4940-bbc8-fc424ff2d285 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.779431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc0349-340b-4940-bbc8-fc424ff2d285 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.784013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0acc1a-5c84-4f43-9a3e-8e9e3eb6fa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.784666 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0acc1a-5c84-4f43-9a3e-8e9e3eb6fa12 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.784929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0acc1a-5c84-4f43-9a3e-8e9e3eb6fa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.785202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0acc1a-5c84-4f43-9a3e-8e9e3eb6fa12 admin admin] PATH_INFO: `/v3/projects/203f73e0bfea4d2e9f10bdc8247150ca/users/09864cbc8fae4bc19fdde59c322dd0d6/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.800083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0acc1a-5c84-4f43-9a3e-8e9e3eb6fa12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=203f73e0bfea4d2e9f10bdc8247150ca, user_id=09864cbc8fae4bc19fdde59c322dd0d6, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.802845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0acc1a-5c84-4f43-9a3e-8e9e3eb6fa12 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.817314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1343/2694] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:56 2022] PUT /identity/v3/projects/203f73e0bfea4d2e9f10bdc8247150ca/users/09864cbc8fae4bc19fdde59c322dd0d6/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:56.821567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604db7c6-16ff-45b5-a9fb-b230b5241de8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.833067 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604db7c6-16ff-45b5-a9fb-b230b5241de8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.890115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e450dda-5a10-4034-9de1-0acdf21b39c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.890727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e450dda-5a10-4034-9de1-0acdf21b39c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.891008 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e450dda-5a10-4034-9de1-0acdf21b39c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.891281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e450dda-5a10-4034-9de1-0acdf21b39c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.891930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e450dda-5a10-4034-9de1-0acdf21b39c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.893575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e450dda-5a10-4034-9de1-0acdf21b39c0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.903180 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1352/2695] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:56.907578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0acc1a-5c84-4f43-9a3e-8e9e3eb6fa12 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:56.918768 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0acc1a-5c84-4f43-9a3e-8e9e3eb6fa12 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:56.923865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11087705-4c0a-45aa-a809-c62c6beafcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:56.924508 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11087705-4c0a-45aa-a809-c62c6beafcfb admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.924782 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11087705-4c0a-45aa-a809-c62c6beafcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.925057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11087705-4c0a-45aa-a809-c62c6beafcfb admin admin] PATH_INFO: `/v3/projects/203f73e0bfea4d2e9f10bdc8247150ca/users/09864cbc8fae4bc19fdde59c322dd0d6/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.926942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11087705-4c0a-45aa-a809-c62c6beafcfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=203f73e0bfea4d2e9f10bdc8247150ca, user_id=09864cbc8fae4bc19fdde59c322dd0d6, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:56.929798 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11087705-4c0a-45aa-a809-c62c6beafcfb admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:56.932195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:56.943028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1344/2696] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:56 2022] PUT /identity/v3/projects/203f73e0bfea4d2e9f10bdc8247150ca/users/09864cbc8fae4bc19fdde59c322dd0d6/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:56.953087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d85bf-5335-4c48-980f-531b8b6fe6cf None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:56.953395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d85bf-5335-4c48-980f-531b8b6fe6cf None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:56.953690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d85bf-5335-4c48-980f-531b8b6fe6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:56.983014 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-a75d85bf-5335-4c48-980f-531b8b6fe6cf None None] MFA Rules not processed for user `09864cbc8fae4bc19fdde59c322dd0d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:56.984738 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a75d85bf-5335-4c48-980f-531b8b6fe6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:57.051085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1353/2697] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:57.063859 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11087705-4c0a-45aa-a809-c62c6beafcfb admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:57.065870 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-11087705-4c0a-45aa-a809-c62c6beafcfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:57.122720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11087705-4c0a-45aa-a809-c62c6beafcfb admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:57.126836 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9384591-d1b3-4f78-8c8e-bef08ce9b895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:57.127430 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9384591-d1b3-4f78-8c8e-bef08ce9b895 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:57.127707 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9384591-d1b3-4f78-8c8e-bef08ce9b895 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:57.127972 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9384591-d1b3-4f78-8c8e-bef08ce9b895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:57.136923 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9384591-d1b3-4f78-8c8e-bef08ce9b895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:57.138561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9384591-d1b3-4f78-8c8e-bef08ce9b895 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:57.149967 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1345/2698] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:57.547889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d85bf-5335-4c48-980f-531b8b6fe6cf None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:57.609787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d85bf-5335-4c48-980f-531b8b6fe6cf None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:57.614271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:57.614894 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:57.615172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:57.615442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] PATH_INFO: `/v3/users/ced98667e3a44b62adea9f4b95667680` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:57.616562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced98667e3a44b62adea9f4b95667680)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:57.618519 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:57.622877 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:57.665261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced98667e3a44b62adea9f4b95667680'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:57.665953 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced98667e3a44b62adea9f4b95667680'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:57.672838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced98667e3a44b62adea9f4b95667680'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:57.685114 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced98667e3a44b62adea9f4b95667680'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:57.692962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced98667e3a44b62adea9f4b95667680'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:57.699304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1354/2699] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:57 2022] DELETE /identity/v3/users/ced98667e3a44b62adea9f4b95667680 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:02:57.704495 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c40d9-0480-4b5b-8688-6ffe41dde7fc None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:57.704845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c40d9-0480-4b5b-8688-6ffe41dde7fc None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:57.705558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c40d9-0480-4b5b-8688-6ffe41dde7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:57.724646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ee8c40d9-0480-4b5b-8688-6ffe41dde7fc None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:57.726073 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ee8c40d9-0480-4b5b-8688-6ffe41dde7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:57.794061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1346/2700] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:02:57.803829 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:57.813345 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:57.871292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece7161-c4de-470e-999b-f75bd4fac758 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:57.874930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b6fc7-9760-4fd0-9f14-fc1dbef7b228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:57.875532 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b6fc7-9760-4fd0-9f14-fc1dbef7b228 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:57.875811 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b6fc7-9760-4fd0-9f14-fc1dbef7b228 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:57.876085 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b6fc7-9760-4fd0-9f14-fc1dbef7b228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:57.884942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b6fc7-9760-4fd0-9f14-fc1dbef7b228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:57.886646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b6fc7-9760-4fd0-9f14-fc1dbef7b228 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:57.896414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1355/2701] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:02:58.182052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c40d9-0480-4b5b-8688-6ffe41dde7fc None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:58.183679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ee8c40d9-0480-4b5b-8688-6ffe41dde7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:58.197153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c40d9-0480-4b5b-8688-6ffe41dde7fc None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:58.201965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:58.203254 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:58.203624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:58.203956 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] PATH_INFO: `/v3/projects/82c536ee58aa410cbd0ef1b6dfe63d7c` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:58.205421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c536ee58aa410cbd0ef1b6dfe63d7c)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:58.207916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:58.254468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c536ee58aa410cbd0ef1b6dfe63d7c', 'request_id': 'req-c4772901-55a6-41a0-b056-757da78c855e'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:58.262294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c536ee58aa410cbd0ef1b6dfe63d7c', 'request_id': 'req-c4772901-55a6-41a0-b056-757da78c855e'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:58.269765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c536ee58aa410cbd0ef1b6dfe63d7c', 'request_id': 'req-c4772901-55a6-41a0-b056-757da78c855e'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:58.282246 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] The token cache is being invalidate because project 82c536ee58aa410cbd0ef1b6dfe63d7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:58.282541 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:58.283337 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:58.284861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1347/2702] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:58 2022] DELETE /identity/v3/projects/82c536ee58aa410cbd0ef1b6dfe63d7c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:58.288833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b6fc7-9760-4fd0-9f14-fc1dbef7b228 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:58.301663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b6fc7-9760-4fd0-9f14-fc1dbef7b228 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:58.357735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:58.358529 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:58.359420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:58.359817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] PATH_INFO: `/v3/users/09864cbc8fae4bc19fdde59c322dd0d6` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:58.361318 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] RBAC: Authorizing `identity:delete_user(user_id=09864cbc8fae4bc19fdde59c322dd0d6)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:58.363527 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:58.395551 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:58.414134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09864cbc8fae4bc19fdde59c322dd0d6'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:58.414799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09864cbc8fae4bc19fdde59c322dd0d6'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:58.422114 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09864cbc8fae4bc19fdde59c322dd0d6'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:58.438365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09864cbc8fae4bc19fdde59c322dd0d6'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:58.445636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d3d458b9-be16-43d1-afaf-8745cdf3c51c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09864cbc8fae4bc19fdde59c322dd0d6'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:58.453223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1356/2703] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:02:58 2022] DELETE /identity/v3/users/09864cbc8fae4bc19fdde59c322dd0d6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:58.976759 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:58.978274 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:58.993170 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4772901-55a6-41a0-b056-757da78c855e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.010599 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:59.053560 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.054300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.054794 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.055149 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] PATH_INFO: `/v3/projects/203f73e0bfea4d2e9f10bdc8247150ca` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.056376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] RBAC: Authorizing `identity:delete_project(project_id=203f73e0bfea4d2e9f10bdc8247150ca)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.058395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.106446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203f73e0bfea4d2e9f10bdc8247150ca', 'request_id': 'req-b64d23ca-b201-4d0b-9c5c-b7726557723d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:59.113727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203f73e0bfea4d2e9f10bdc8247150ca', 'request_id': 'req-b64d23ca-b201-4d0b-9c5c-b7726557723d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:59.120067 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203f73e0bfea4d2e9f10bdc8247150ca', 'request_id': 'req-b64d23ca-b201-4d0b-9c5c-b7726557723d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:59.130044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] The token cache is being invalidate because project 203f73e0bfea4d2e9f10bdc8247150ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:02:59.130325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:59.130932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-b64d23ca-b201-4d0b-9c5c-b7726557723d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:02:59.132428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1348/2704] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:02:58 2022] DELETE /identity/v3/projects/203f73e0bfea4d2e9f10bdc8247150ca => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:59.138123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52560d4b-58ef-4871-b1f4-0a246046b8d4 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.138611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52560d4b-58ef-4871-b1f4-0a246046b8d4 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.138724 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52560d4b-58ef-4871-b1f4-0a246046b8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.161712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-52560d4b-58ef-4871-b1f4-0a246046b8d4 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:59.162930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-52560d4b-58ef-4871-b1f4-0a246046b8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:59.222577 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1357/2705] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:59.230199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cab40-cb90-4cae-8740-050f711cca71 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.230664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cab40-cb90-4cae-8740-050f711cca71 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.230792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cab40-cb90-4cae-8740-050f711cca71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.251637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-3e2cab40-cb90-4cae-8740-050f711cca71 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:59.253239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-3e2cab40-cb90-4cae-8740-050f711cca71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:59.261355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1349/2706] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:59.266172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52560d4b-58ef-4871-b1f4-0a246046b8d4 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.275793 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52560d4b-58ef-4871-b1f4-0a246046b8d4 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.280381 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce7a19-f821-4eeb-8a80-994c4317c38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.281558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce7a19-f821-4eeb-8a80-994c4317c38c admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.281847 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce7a19-f821-4eeb-8a80-994c4317c38c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.282123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce7a19-f821-4eeb-8a80-994c4317c38c admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.282936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce7a19-f821-4eeb-8a80-994c4317c38c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:02:59.283250 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce7a19-f821-4eeb-8a80-994c4317c38c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.284889 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce7a19-f821-4eeb-8a80-994c4317c38c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.298499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1358/2707] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:02:59.302985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cab40-cb90-4cae-8740-050f711cca71 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.313239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cab40-cb90-4cae-8740-050f711cca71 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.318047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d53706-f356-43bb-893f-6df81b31a054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.318610 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d53706-f356-43bb-893f-6df81b31a054 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.319050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d53706-f356-43bb-893f-6df81b31a054 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.319164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d53706-f356-43bb-893f-6df81b31a054 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.319792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d53706-f356-43bb-893f-6df81b31a054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.321709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d53706-f356-43bb-893f-6df81b31a054 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.352999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1350/2708] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:59 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:59.357359 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce7a19-f821-4eeb-8a80-994c4317c38c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.368734 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce7a19-f821-4eeb-8a80-994c4317c38c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.425017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036dd1b5-31de-4ef3-98ac-ea49676c588c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.425660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036dd1b5-31de-4ef3-98ac-ea49676c588c admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.425939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036dd1b5-31de-4ef3-98ac-ea49676c588c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.426210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036dd1b5-31de-4ef3-98ac-ea49676c588c admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.426843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036dd1b5-31de-4ef3-98ac-ea49676c588c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.428780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036dd1b5-31de-4ef3-98ac-ea49676c588c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.450632 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1359/2709] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:59 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:59.455670 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d53706-f356-43bb-893f-6df81b31a054 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.465958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d53706-f356-43bb-893f-6df81b31a054 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.470701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b752fa2-cab4-4563-8877-c9382a265caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.471300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b752fa2-cab4-4563-8877-c9382a265caf admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.471582 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b752fa2-cab4-4563-8877-c9382a265caf admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.471855 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b752fa2-cab4-4563-8877-c9382a265caf admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.472497 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b752fa2-cab4-4563-8877-c9382a265caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.474217 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b752fa2-cab4-4563-8877-c9382a265caf admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.488000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1351/2710] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:59.491985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036dd1b5-31de-4ef3-98ac-ea49676c588c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.502362 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036dd1b5-31de-4ef3-98ac-ea49676c588c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.506974 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827d77-1cd6-452f-9e8b-277825ab6064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.507565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827d77-1cd6-452f-9e8b-277825ab6064 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.507850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827d77-1cd6-452f-9e8b-277825ab6064 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.508130 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827d77-1cd6-452f-9e8b-277825ab6064 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.508782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13827d77-1cd6-452f-9e8b-277825ab6064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.510425 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13827d77-1cd6-452f-9e8b-277825ab6064 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.521718 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1360/2711] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:02:59.525540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b752fa2-cab4-4563-8877-c9382a265caf admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.535377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b752fa2-cab4-4563-8877-c9382a265caf admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.540135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce702aa-614a-4bea-8575-856bab6f1157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.540771 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce702aa-614a-4bea-8575-856bab6f1157 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.541049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce702aa-614a-4bea-8575-856bab6f1157 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.541321 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce702aa-614a-4bea-8575-856bab6f1157 admin admin] PATH_INFO: `/v3/projects/52b2434ff97a4454af7b136220921e85/users/40d64aa3f0854f1b888110db3b555915/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.555622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce702aa-614a-4bea-8575-856bab6f1157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b2434ff97a4454af7b136220921e85, user_id=40d64aa3f0854f1b888110db3b555915, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.558377 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce702aa-614a-4bea-8575-856bab6f1157 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.571524 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1352/2712] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:59 2022] PUT /identity/v3/projects/52b2434ff97a4454af7b136220921e85/users/40d64aa3f0854f1b888110db3b555915/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:59.583716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c2ab0-391e-4d89-8a62-b12f97a3ec07 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.584224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c2ab0-391e-4d89-8a62-b12f97a3ec07 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.584518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c2ab0-391e-4d89-8a62-b12f97a3ec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.610939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:59.620050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-1e5c2ab0-391e-4d89-8a62-b12f97a3ec07 None None] MFA Rules not processed for user `40d64aa3f0854f1b888110db3b555915`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:02:59.621915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1e5c2ab0-391e-4d89-8a62-b12f97a3ec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:02:59.685885 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1361/2713] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:02:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:02:59.690516 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce702aa-614a-4bea-8575-856bab6f1157 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.738043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:02:59.756971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce702aa-614a-4bea-8575-856bab6f1157 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.761588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5bea2-f9cf-489a-ac58-9f4f758d297e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.762188 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5bea2-f9cf-489a-ac58-9f4f758d297e admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.762464 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5bea2-f9cf-489a-ac58-9f4f758d297e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.762734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5bea2-f9cf-489a-ac58-9f4f758d297e admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.763369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5bea2-f9cf-489a-ac58-9f4f758d297e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.765286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5bea2-f9cf-489a-ac58-9f4f758d297e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.793152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1353/2714] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:02:59 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:02:59.797636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c2ab0-391e-4d89-8a62-b12f97a3ec07 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.808458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c2ab0-391e-4d89-8a62-b12f97a3ec07 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.874050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111d910-efe4-4a47-9523-242000e1340a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.874730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111d910-efe4-4a47-9523-242000e1340a admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.875039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111d910-efe4-4a47-9523-242000e1340a admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.875316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111d910-efe4-4a47-9523-242000e1340a admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.875989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111d910-efe4-4a47-9523-242000e1340a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.877939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111d910-efe4-4a47-9523-242000e1340a admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.898901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1362/2715] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:02:59 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:02:59.902968 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5bea2-f9cf-489a-ac58-9f4f758d297e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.913936 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5bea2-f9cf-489a-ac58-9f4f758d297e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.918538 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512298ae-c74e-4cad-bfc8-24e500a40748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.919135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512298ae-c74e-4cad-bfc8-24e500a40748 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.919446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512298ae-c74e-4cad-bfc8-24e500a40748 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.919717 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512298ae-c74e-4cad-bfc8-24e500a40748 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.920389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512298ae-c74e-4cad-bfc8-24e500a40748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.922137 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512298ae-c74e-4cad-bfc8-24e500a40748 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.933336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1354/2716] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:02:59.937715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111d910-efe4-4a47-9523-242000e1340a admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.947716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111d910-efe4-4a47-9523-242000e1340a admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:02:59.952407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104db9e-e144-486b-907e-c9101fbbb8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:02:59.953585 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104db9e-e144-486b-907e-c9101fbbb8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:02:59.953875 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104db9e-e144-486b-907e-c9101fbbb8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:02:59.954152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104db9e-e144-486b-907e-c9101fbbb8e9 admin admin] PATH_INFO: `/v3/projects/240565e9b8b04bec8f94972d414fc6f2/users/bed7ff003a47448bbaedd8cc73e645e4/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:02:59.969204 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104db9e-e144-486b-907e-c9101fbbb8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240565e9b8b04bec8f94972d414fc6f2, user_id=bed7ff003a47448bbaedd8cc73e645e4, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:02:59.971902 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104db9e-e144-486b-907e-c9101fbbb8e9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:02:59.985050 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1363/2717] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:02:59 2022] PUT /identity/v3/projects/240565e9b8b04bec8f94972d414fc6f2/users/bed7ff003a47448bbaedd8cc73e645e4/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:02:59.989012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512298ae-c74e-4cad-bfc8-24e500a40748 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:02:59.998812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512298ae-c74e-4cad-bfc8-24e500a40748 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:00.056001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66726d7d-c6b9-4cb2-9162-08d0735b5330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:00.056605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726d7d-c6b9-4cb2-9162-08d0735b5330 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:00.056881 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726d7d-c6b9-4cb2-9162-08d0735b5330 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:00.057157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726d7d-c6b9-4cb2-9162-08d0735b5330 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:00.057837 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66726d7d-c6b9-4cb2-9162-08d0735b5330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:00.059422 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66726d7d-c6b9-4cb2-9162-08d0735b5330 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:00.069544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1355/2718] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:00.073739 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104db9e-e144-486b-907e-c9101fbbb8e9 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:00.084038 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104db9e-e144-486b-907e-c9101fbbb8e9 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:00.088630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645972e0-42b7-4103-8b41-026ab22cc37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:00.089275 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645972e0-42b7-4103-8b41-026ab22cc37e admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:00.089603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645972e0-42b7-4103-8b41-026ab22cc37e admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:00.089884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645972e0-42b7-4103-8b41-026ab22cc37e admin admin] PATH_INFO: `/v3/projects/240565e9b8b04bec8f94972d414fc6f2/users/bed7ff003a47448bbaedd8cc73e645e4/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:00.091826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645972e0-42b7-4103-8b41-026ab22cc37e admin admin] RBAC: Authorizing `identity:create_grant(project_id=240565e9b8b04bec8f94972d414fc6f2, user_id=bed7ff003a47448bbaedd8cc73e645e4, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:00.094498 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645972e0-42b7-4103-8b41-026ab22cc37e admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:00.104735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1364/2719] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:00 2022] PUT /identity/v3/projects/240565e9b8b04bec8f94972d414fc6f2/users/bed7ff003a47448bbaedd8cc73e645e4/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:00.115510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c39b1-fb32-451c-909e-58ec53b48d96 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:00.115791 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c39b1-fb32-451c-909e-58ec53b48d96 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:00.116060 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c39b1-fb32-451c-909e-58ec53b48d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:00.145918 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-cc3c39b1-fb32-451c-909e-58ec53b48d96 None None] MFA Rules not processed for user `bed7ff003a47448bbaedd8cc73e645e4`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:00.147167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-cc3c39b1-fb32-451c-909e-58ec53b48d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:00.228025 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1356/2720] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:00.238658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645972e0-42b7-4103-8b41-026ab22cc37e admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:00.240477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-645972e0-42b7-4103-8b41-026ab22cc37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:00.299592 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645972e0-42b7-4103-8b41-026ab22cc37e admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:00.303253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299ea92-d90c-40d8-ba5d-de468bc2f84b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:00.303858 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299ea92-d90c-40d8-ba5d-de468bc2f84b service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:00.304136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299ea92-d90c-40d8-ba5d-de468bc2f84b service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:00.304409 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299ea92-d90c-40d8-ba5d-de468bc2f84b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:00.309468 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:00.361071 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299ea92-d90c-40d8-ba5d-de468bc2f84b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:00.362547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299ea92-d90c-40d8-ba5d-de468bc2f84b service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:00.371387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1365/2721] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:10.225105 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=80,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:10.367358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=24 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:16.010645 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c39b1-fb32-451c-909e-58ec53b48d96 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:16.012775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-cc3c39b1-fb32-451c-909e-58ec53b48d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:16.073222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c39b1-fb32-451c-909e-58ec53b48d96 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:16.076729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f30b0-3ec6-4c0b-beed-e7f773e2758f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:16.077329 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f30b0-3ec6-4c0b-beed-e7f773e2758f service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:16.077634 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f30b0-3ec6-4c0b-beed-e7f773e2758f service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:16.077905 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f30b0-3ec6-4c0b-beed-e7f773e2758f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:16.079585 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-324f30b0-3ec6-4c0b-beed-e7f773e2758f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:16.089293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f30b0-3ec6-4c0b-beed-e7f773e2758f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:16.090745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f30b0-3ec6-4c0b-beed-e7f773e2758f service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:16.098920 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1357/2722] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:03:18.561199 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299ea92-d90c-40d8-ba5d-de468bc2f84b service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:18.623798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299ea92-d90c-40d8-ba5d-de468bc2f84b service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:18.629290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:18.630004 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:18.630277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:18.630544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] PATH_INFO: `/v3/users/40d64aa3f0854f1b888110db3b555915` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:18.631691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=40d64aa3f0854f1b888110db3b555915)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:18.633624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:18.682320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40d64aa3f0854f1b888110db3b555915'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:18.683376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40d64aa3f0854f1b888110db3b555915'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:18.691385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40d64aa3f0854f1b888110db3b555915'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:18.703518 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40d64aa3f0854f1b888110db3b555915'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:18.711294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40d64aa3f0854f1b888110db3b555915'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:18.717316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1366/2723] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:18 2022] DELETE /identity/v3/users/40d64aa3f0854f1b888110db3b555915 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:18.721775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac838ee1-5cd5-4b75-b3f2-acfac3723086 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:18.722306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac838ee1-5cd5-4b75-b3f2-acfac3723086 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:18.722306 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac838ee1-5cd5-4b75-b3f2-acfac3723086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:18.742185 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-ac838ee1-5cd5-4b75-b3f2-acfac3723086 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:18.743378 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ac838ee1-5cd5-4b75-b3f2-acfac3723086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:18.800732 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1358/2724] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:18.810541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:18.811938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:18.871230 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16c8fb-cbda-4b06-8f32-4a6b4c65c4ee admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:18.875499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e807db-6930-41ac-a24a-4ca3843b48b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:18.876102 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e807db-6930-41ac-a24a-4ca3843b48b1 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:18.876395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e807db-6930-41ac-a24a-4ca3843b48b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:18.876654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e807db-6930-41ac-a24a-4ca3843b48b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:18.885509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e807db-6930-41ac-a24a-4ca3843b48b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:18.886930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e807db-6930-41ac-a24a-4ca3843b48b1 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:18.896521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1367/2725] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:03:19.173950 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac838ee1-5cd5-4b75-b3f2-acfac3723086 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:19.175381 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ac838ee1-5cd5-4b75-b3f2-acfac3723086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:19.188711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac838ee1-5cd5-4b75-b3f2-acfac3723086 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:19.193317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:19.194038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:19.194389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:19.194727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] PATH_INFO: `/v3/projects/52b2434ff97a4454af7b136220921e85` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:19.196090 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52b2434ff97a4454af7b136220921e85)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:19.197952 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:19.249132 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b2434ff97a4454af7b136220921e85', 'request_id': 'req-e65ff44a-90b9-49d3-b551-9a27465c6996'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.257603 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b2434ff97a4454af7b136220921e85', 'request_id': 'req-e65ff44a-90b9-49d3-b551-9a27465c6996'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.263986 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b2434ff97a4454af7b136220921e85', 'request_id': 'req-e65ff44a-90b9-49d3-b551-9a27465c6996'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.275042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] The token cache is being invalidate because project 52b2434ff97a4454af7b136220921e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:19.275336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.276043 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.277827 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1359/2726] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:19 2022] DELETE /identity/v3/projects/52b2434ff97a4454af7b136220921e85 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:19.282523 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e807db-6930-41ac-a24a-4ca3843b48b1 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:19.293107 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e807db-6930-41ac-a24a-4ca3843b48b1 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:19.314991 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:19.355992 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:19.356619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:19.356896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:19.357165 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] PATH_INFO: `/v3/users/bed7ff003a47448bbaedd8cc73e645e4` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:19.358302 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bed7ff003a47448bbaedd8cc73e645e4)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:19.361298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:19.414860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bed7ff003a47448bbaedd8cc73e645e4'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.415609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bed7ff003a47448bbaedd8cc73e645e4'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.422892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bed7ff003a47448bbaedd8cc73e645e4'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.436284 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bed7ff003a47448bbaedd8cc73e645e4'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.447439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4cc74206-97e7-4057-a8ec-9316e0b863dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bed7ff003a47448bbaedd8cc73e645e4'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.454069 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1368/2727] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:19 2022] DELETE /identity/v3/users/bed7ff003a47448bbaedd8cc73e645e4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:19.739086 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:19.740646 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:19.755335 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ff44a-90b9-49d3-b551-9a27465c6996 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:19.786862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:19.823699 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:19.824328 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:19.824602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:19.824867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] PATH_INFO: `/v3/projects/240565e9b8b04bec8f94972d414fc6f2` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:19.825993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=240565e9b8b04bec8f94972d414fc6f2)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:19.827917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:19.874355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240565e9b8b04bec8f94972d414fc6f2', 'request_id': 'req-5c302c9b-d36f-4172-948a-f272323b2f95'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.881489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240565e9b8b04bec8f94972d414fc6f2', 'request_id': 'req-5c302c9b-d36f-4172-948a-f272323b2f95'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.887746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240565e9b8b04bec8f94972d414fc6f2', 'request_id': 'req-5c302c9b-d36f-4172-948a-f272323b2f95'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.898074 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] The token cache is being invalidate because project 240565e9b8b04bec8f94972d414fc6f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:19.898357 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.899020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5c302c9b-d36f-4172-948a-f272323b2f95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:19.900514 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1360/2728] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:19 2022] DELETE /identity/v3/projects/240565e9b8b04bec8f94972d414fc6f2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:19.908337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee16e76-729d-4a59-9e9f-68e6e0898a6f None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:19.908807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee16e76-729d-4a59-9e9f-68e6e0898a6f None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:19.908907 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee16e76-729d-4a59-9e9f-68e6e0898a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:19.928514 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6ee16e76-729d-4a59-9e9f-68e6e0898a6f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:19.929721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6ee16e76-729d-4a59-9e9f-68e6e0898a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:19.990923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1369/2729] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:19.999177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5fb14-1ef2-4d4d-881f-d62d2551f7da None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:19.999309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5fb14-1ef2-4d4d-881f-d62d2551f7da None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:19.999474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5fb14-1ef2-4d4d-881f-d62d2551f7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.019263 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d1a5fb14-1ef2-4d4d-881f-d62d2551f7da None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:20.020449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d1a5fb14-1ef2-4d4d-881f-d62d2551f7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:20.029901 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1361/2730] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:20.034662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee16e76-729d-4a59-9e9f-68e6e0898a6f None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.044442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee16e76-729d-4a59-9e9f-68e6e0898a6f None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.050365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97db531-fc8a-4d4c-be6a-a494cb53fd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.050949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97db531-fc8a-4d4c-be6a-a494cb53fd15 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.051227 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97db531-fc8a-4d4c-be6a-a494cb53fd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.051497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97db531-fc8a-4d4c-be6a-a494cb53fd15 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.052160 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97db531-fc8a-4d4c-be6a-a494cb53fd15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:03:20.052461 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97db531-fc8a-4d4c-be6a-a494cb53fd15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.054121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97db531-fc8a-4d4c-be6a-a494cb53fd15 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.066189 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1370/2731] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:03:20.070733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5fb14-1ef2-4d4d-881f-d62d2551f7da None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.079824 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5fb14-1ef2-4d4d-881f-d62d2551f7da None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.084735 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9254b2b-dbc7-4909-a3b3-20feb9630c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.085326 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9254b2b-dbc7-4909-a3b3-20feb9630c00 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.085638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9254b2b-dbc7-4909-a3b3-20feb9630c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.085907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9254b2b-dbc7-4909-a3b3-20feb9630c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.086531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9254b2b-dbc7-4909-a3b3-20feb9630c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.088417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9254b2b-dbc7-4909-a3b3-20feb9630c00 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.118045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1362/2732] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:03:20 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:20.122174 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97db531-fc8a-4d4c-be6a-a494cb53fd15 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.132192 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97db531-fc8a-4d4c-be6a-a494cb53fd15 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.158121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:20.192260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291256eb-0591-42b6-8a9d-9ac3f622d712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.192260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291256eb-0591-42b6-8a9d-9ac3f622d712 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.192260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291256eb-0591-42b6-8a9d-9ac3f622d712 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.192260 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291256eb-0591-42b6-8a9d-9ac3f622d712 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.192842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291256eb-0591-42b6-8a9d-9ac3f622d712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.194397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291256eb-0591-42b6-8a9d-9ac3f622d712 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.216496 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1371/2733] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:03:20 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:03:20.221202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9254b2b-dbc7-4909-a3b3-20feb9630c00 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.231317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9254b2b-dbc7-4909-a3b3-20feb9630c00 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.236147 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeb33a-b159-4124-8549-c0cc27bd8b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.236734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeb33a-b159-4124-8549-c0cc27bd8b84 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.237010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeb33a-b159-4124-8549-c0cc27bd8b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.237278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeb33a-b159-4124-8549-c0cc27bd8b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.237947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eeb33a-b159-4124-8549-c0cc27bd8b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.239535 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eeb33a-b159-4124-8549-c0cc27bd8b84 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.249225 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1363/2734] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:20.252904 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291256eb-0591-42b6-8a9d-9ac3f622d712 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.263252 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291256eb-0591-42b6-8a9d-9ac3f622d712 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.268337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3603f-ef6e-4031-9e50-5697816ce690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.268930 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3603f-ef6e-4031-9e50-5697816ce690 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.269213 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3603f-ef6e-4031-9e50-5697816ce690 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.269484 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3603f-ef6e-4031-9e50-5697816ce690 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.270152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3603f-ef6e-4031-9e50-5697816ce690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.271793 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3603f-ef6e-4031-9e50-5697816ce690 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.288358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1372/2735] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:20.292521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeb33a-b159-4124-8549-c0cc27bd8b84 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.302613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeb33a-b159-4124-8549-c0cc27bd8b84 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.307363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7eab6c-d294-477f-9a81-b64e3efd4057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.308004 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7eab6c-d294-477f-9a81-b64e3efd4057 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.308280 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7eab6c-d294-477f-9a81-b64e3efd4057 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.308546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7eab6c-d294-477f-9a81-b64e3efd4057 admin admin] PATH_INFO: `/v3/projects/3b5afce05b9e4bc7956b0545be8d5815/users/9e5c966dc3a94ce0a959d2c9b42e03fc/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.322953 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7eab6c-d294-477f-9a81-b64e3efd4057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b5afce05b9e4bc7956b0545be8d5815, user_id=9e5c966dc3a94ce0a959d2c9b42e03fc, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.325632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7eab6c-d294-477f-9a81-b64e3efd4057 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.339167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1364/2736] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:20 2022] PUT /identity/v3/projects/3b5afce05b9e4bc7956b0545be8d5815/users/9e5c966dc3a94ce0a959d2c9b42e03fc/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:20.351904 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283062fb-cda6-4752-9a19-eccec1b5cde7 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.352188 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283062fb-cda6-4752-9a19-eccec1b5cde7 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.352463 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283062fb-cda6-4752-9a19-eccec1b5cde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.380101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-283062fb-cda6-4752-9a19-eccec1b5cde7 None None] MFA Rules not processed for user `9e5c966dc3a94ce0a959d2c9b42e03fc`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:20.381307 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-283062fb-cda6-4752-9a19-eccec1b5cde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:20.447606 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1373/2737] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:20.453391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7eab6c-d294-477f-9a81-b64e3efd4057 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.505091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:20.517269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7eab6c-d294-477f-9a81-b64e3efd4057 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.522292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e8a9e-d6f8-4e21-a4fa-802496c5c002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.522897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e8a9e-d6f8-4e21-a4fa-802496c5c002 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.523175 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e8a9e-d6f8-4e21-a4fa-802496c5c002 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.523436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e8a9e-d6f8-4e21-a4fa-802496c5c002 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.524063 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e8a9e-d6f8-4e21-a4fa-802496c5c002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.525991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e8a9e-d6f8-4e21-a4fa-802496c5c002 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.554062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1365/2738] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:03:20 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:20.558223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283062fb-cda6-4752-9a19-eccec1b5cde7 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.569144 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283062fb-cda6-4752-9a19-eccec1b5cde7 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.623540 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc5b5c-7ee9-4ab7-a788-e2cf4389cf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.624140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5b5c-7ee9-4ab7-a788-e2cf4389cf44 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.624421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5b5c-7ee9-4ab7-a788-e2cf4389cf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.625185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5b5c-7ee9-4ab7-a788-e2cf4389cf44 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.625850 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc5b5c-7ee9-4ab7-a788-e2cf4389cf44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.627753 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc5b5c-7ee9-4ab7-a788-e2cf4389cf44 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.648855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1374/2739] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:03:20 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:03:20.652926 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e8a9e-d6f8-4e21-a4fa-802496c5c002 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.663156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e8a9e-d6f8-4e21-a4fa-802496c5c002 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.667622 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb553a5-22af-42dc-97f8-2ef6f29f49d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.668215 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb553a5-22af-42dc-97f8-2ef6f29f49d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.668489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb553a5-22af-42dc-97f8-2ef6f29f49d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.668754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb553a5-22af-42dc-97f8-2ef6f29f49d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.669383 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb553a5-22af-42dc-97f8-2ef6f29f49d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.670998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb553a5-22af-42dc-97f8-2ef6f29f49d9 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.681235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1366/2740] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:20.685129 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc5b5c-7ee9-4ab7-a788-e2cf4389cf44 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.695277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc5b5c-7ee9-4ab7-a788-e2cf4389cf44 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.699838 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1d7c9-87b4-44ba-b0d2-95b02476d9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.700495 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1d7c9-87b4-44ba-b0d2-95b02476d9b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.700774 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1d7c9-87b4-44ba-b0d2-95b02476d9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.701044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1d7c9-87b4-44ba-b0d2-95b02476d9b8 admin admin] PATH_INFO: `/v3/projects/ba6dff63b41f43d6a6bea6c250cc381d/users/b940523b69ff42cdb3661b1d81263aef/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.716952 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1d7c9-87b4-44ba-b0d2-95b02476d9b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba6dff63b41f43d6a6bea6c250cc381d, user_id=b940523b69ff42cdb3661b1d81263aef, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.719634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1d7c9-87b4-44ba-b0d2-95b02476d9b8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.732722 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1375/2741] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:20 2022] PUT /identity/v3/projects/ba6dff63b41f43d6a6bea6c250cc381d/users/b940523b69ff42cdb3661b1d81263aef/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:20.736601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb553a5-22af-42dc-97f8-2ef6f29f49d9 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.746359 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb553a5-22af-42dc-97f8-2ef6f29f49d9 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.806059 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88a62a-b4c5-4bba-a1f9-de0b7b121d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.806664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a62a-b4c5-4bba-a1f9-de0b7b121d5e admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.806942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a62a-b4c5-4bba-a1f9-de0b7b121d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.807213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a62a-b4c5-4bba-a1f9-de0b7b121d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.807846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88a62a-b4c5-4bba-a1f9-de0b7b121d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.809434 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88a62a-b4c5-4bba-a1f9-de0b7b121d5e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.819406 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1367/2742] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:03:20.823584 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1d7c9-87b4-44ba-b0d2-95b02476d9b8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.829685 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:20.836136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1d7c9-87b4-44ba-b0d2-95b02476d9b8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:20.840999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a02f48-8d2a-4b34-affb-6998f203957d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:20.841723 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a02f48-8d2a-4b34-affb-6998f203957d admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.842016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a02f48-8d2a-4b34-affb-6998f203957d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.842277 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a02f48-8d2a-4b34-affb-6998f203957d admin admin] PATH_INFO: `/v3/projects/ba6dff63b41f43d6a6bea6c250cc381d/users/b940523b69ff42cdb3661b1d81263aef/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.844162 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a02f48-8d2a-4b34-affb-6998f203957d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba6dff63b41f43d6a6bea6c250cc381d, user_id=b940523b69ff42cdb3661b1d81263aef, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:20.846799 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a02f48-8d2a-4b34-affb-6998f203957d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:20.856840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1376/2743] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:20 2022] PUT /identity/v3/projects/ba6dff63b41f43d6a6bea6c250cc381d/users/b940523b69ff42cdb3661b1d81263aef/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:20.867817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d4071-4080-4ff1-a877-92331d8860ef None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:20.868098 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d4071-4080-4ff1-a877-92331d8860ef None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:20.868370 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d4071-4080-4ff1-a877-92331d8860ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:20.895489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-834d4071-4080-4ff1-a877-92331d8860ef None None] MFA Rules not processed for user `b940523b69ff42cdb3661b1d81263aef`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:20.896693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-834d4071-4080-4ff1-a877-92331d8860ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:20.964189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1368/2744] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:20.974831 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a02f48-8d2a-4b34-affb-6998f203957d admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:20.976511 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b1a02f48-8d2a-4b34-affb-6998f203957d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:21.031610 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a02f48-8d2a-4b34-affb-6998f203957d admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:21.035245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ae648-955b-466d-8e92-66a6a714612e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:21.035842 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ae648-955b-466d-8e92-66a6a714612e service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:21.036166 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ae648-955b-466d-8e92-66a6a714612e service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:21.036368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ae648-955b-466d-8e92-66a6a714612e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:21.045288 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ae648-955b-466d-8e92-66a6a714612e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:21.047264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ae648-955b-466d-8e92-66a6a714612e service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:21.057153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1377/2745] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:21.599701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d4071-4080-4ff1-a877-92331d8860ef None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:21.647160 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:21.662097 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d4071-4080-4ff1-a877-92331d8860ef None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:21.666589 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:21.667212 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:21.667489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:21.667758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] PATH_INFO: `/v3/users/9e5c966dc3a94ce0a959d2c9b42e03fc` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:21.668873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5c966dc3a94ce0a959d2c9b42e03fc)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:21.670814 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:21.715916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5c966dc3a94ce0a959d2c9b42e03fc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:21.716492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5c966dc3a94ce0a959d2c9b42e03fc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:21.723916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5c966dc3a94ce0a959d2c9b42e03fc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:21.731391 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5c966dc3a94ce0a959d2c9b42e03fc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:21.732051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5c966dc3a94ce0a959d2c9b42e03fc'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:21.748805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1369/2746] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:21 2022] DELETE /identity/v3/users/9e5c966dc3a94ce0a959d2c9b42e03fc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:21.753308 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3aefa-c239-4932-afcb-5d2f28549230 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:21.753633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3aefa-c239-4932-afcb-5d2f28549230 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:21.753971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3aefa-c239-4932-afcb-5d2f28549230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:21.772764 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f7a3aefa-c239-4932-afcb-5d2f28549230 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:21.774022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f7a3aefa-c239-4932-afcb-5d2f28549230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:21.830072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1378/2747] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:03:21.839961 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:21.841364 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:21.899449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d2a5b-e1bb-4e53-b88c-10f43ccbbfc3 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:21.903098 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bef9b-4b66-41a1-bcd5-70e04f6d303f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:21.903698 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bef9b-4b66-41a1-bcd5-70e04f6d303f service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:21.903976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bef9b-4b66-41a1-bcd5-70e04f6d303f service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:21.904248 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bef9b-4b66-41a1-bcd5-70e04f6d303f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:21.913786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bef9b-4b66-41a1-bcd5-70e04f6d303f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:21.915216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bef9b-4b66-41a1-bcd5-70e04f6d303f service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:21.924522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1370/2748] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:22.210070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3aefa-c239-4932-afcb-5d2f28549230 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:22.211581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f7a3aefa-c239-4932-afcb-5d2f28549230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:22.229901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3aefa-c239-4932-afcb-5d2f28549230 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:22.241285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:22.242040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:22.242393 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:22.242761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] PATH_INFO: `/v3/projects/3b5afce05b9e4bc7956b0545be8d5815` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:22.243984 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b5afce05b9e4bc7956b0545be8d5815)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:22.246086 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:22.298360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b5afce05b9e4bc7956b0545be8d5815', 'request_id': 'req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.307508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b5afce05b9e4bc7956b0545be8d5815', 'request_id': 'req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.314840 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b5afce05b9e4bc7956b0545be8d5815', 'request_id': 'req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.325300 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] The token cache is being invalidate because project 3b5afce05b9e4bc7956b0545be8d5815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:22.325627 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.326268 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.327727 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1379/2749] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:22 2022] DELETE /identity/v3/projects/3b5afce05b9e4bc7956b0545be8d5815 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:22.332648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bef9b-4b66-41a1-bcd5-70e04f6d303f service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:22.342292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bef9b-4b66-41a1-bcd5-70e04f6d303f service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:22.399057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:22.399680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:22.399955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:22.400349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] PATH_INFO: `/v3/users/b940523b69ff42cdb3661b1d81263aef` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:22.401472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b940523b69ff42cdb3661b1d81263aef)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:22.403493 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:22.416660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:22.451503 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b940523b69ff42cdb3661b1d81263aef'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.452153 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b940523b69ff42cdb3661b1d81263aef'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.459528 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b940523b69ff42cdb3661b1d81263aef'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.468292 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b940523b69ff42cdb3661b1d81263aef'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.468942 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-7ff32c0d-6a6f-44b7-a1c3-d93b9b59ec7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b940523b69ff42cdb3661b1d81263aef'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.486729 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1371/2750] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:22 2022] DELETE /identity/v3/users/b940523b69ff42cdb3661b1d81263aef => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:22.765019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:22.766673 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:22.782229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d7aae-8b63-46ec-9ac0-f4f6a78677ac admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:22.796237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:22.845176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:22.845855 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:22.846149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:22.846424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] PATH_INFO: `/v3/projects/ba6dff63b41f43d6a6bea6c250cc381d` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:22.847547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba6dff63b41f43d6a6bea6c250cc381d)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:22.849602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:22.896890 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba6dff63b41f43d6a6bea6c250cc381d', 'request_id': 'req-f65aba8f-ce45-43e9-bcc3-cda7f378834c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.905711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba6dff63b41f43d6a6bea6c250cc381d', 'request_id': 'req-f65aba8f-ce45-43e9-bcc3-cda7f378834c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.912765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba6dff63b41f43d6a6bea6c250cc381d', 'request_id': 'req-f65aba8f-ce45-43e9-bcc3-cda7f378834c'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.924366 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] The token cache is being invalidate because project ba6dff63b41f43d6a6bea6c250cc381d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:22.924654 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.925299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-f65aba8f-ce45-43e9-bcc3-cda7f378834c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:22.926796 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1380/2751] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:22 2022] DELETE /identity/v3/projects/ba6dff63b41f43d6a6bea6c250cc381d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:22.931546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759b347-6cdb-42f1-bf5b-4b6f0cdd2d4d None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:22.931997 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759b347-6cdb-42f1-bf5b-4b6f0cdd2d4d None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:22.932100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759b347-6cdb-42f1-bf5b-4b6f0cdd2d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:22.951614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f759b347-6cdb-42f1-bf5b-4b6f0cdd2d4d None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:22.952823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f759b347-6cdb-42f1-bf5b-4b6f0cdd2d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:23.012083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1372/2752] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:23.021147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69237752-6a17-48f5-bbb8-1c1a7aa415f4 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.021436 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69237752-6a17-48f5-bbb8-1c1a7aa415f4 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.021741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69237752-6a17-48f5-bbb8-1c1a7aa415f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.044936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-69237752-6a17-48f5-bbb8-1c1a7aa415f4 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:23.046338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-69237752-6a17-48f5-bbb8-1c1a7aa415f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:23.055449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1381/2753] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:23.059627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759b347-6cdb-42f1-bf5b-4b6f0cdd2d4d None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:23.070121 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759b347-6cdb-42f1-bf5b-4b6f0cdd2d4d None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:23.075362 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb75ee-09cb-4b22-be81-49a4206c0307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:23.076064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb75ee-09cb-4b22-be81-49a4206c0307 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.076436 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb75ee-09cb-4b22-be81-49a4206c0307 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.076841 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb75ee-09cb-4b22-be81-49a4206c0307 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.086788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69237752-6a17-48f5-bbb8-1c1a7aa415f4 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:23.088124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-69237752-6a17-48f5-bbb8-1c1a7aa415f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:23.088873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb75ee-09cb-4b22-be81-49a4206c0307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:03:23.089075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb75ee-09cb-4b22-be81-49a4206c0307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:23.091294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb75ee-09cb-4b22-be81-49a4206c0307 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:23.112382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1373/2754] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:03:23.113402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb75ee-09cb-4b22-be81-49a4206c0307 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:23.126550 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb75ee-09cb-4b22-be81-49a4206c0307 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:23.132640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d9545-6a6f-4e04-9dd3-0a12648b339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:23.133681 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d9545-6a6f-4e04-9dd3-0a12648b339e admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.134100 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d9545-6a6f-4e04-9dd3-0a12648b339e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.134502 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d9545-6a6f-4e04-9dd3-0a12648b339e admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.135344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d9545-6a6f-4e04-9dd3-0a12648b339e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:23.137415 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d9545-6a6f-4e04-9dd3-0a12648b339e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:23.178998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1374/2755] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:03:23 2022] POST /identity/v3/projects => generated 361 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:23.180342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d9545-6a6f-4e04-9dd3-0a12648b339e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:23.207624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69237752-6a17-48f5-bbb8-1c1a7aa415f4 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:23.253224 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:23.260448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d9545-6a6f-4e04-9dd3-0a12648b339e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:23.270441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa9af-395a-4580-b9f1-2fc58701971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:23.270441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa9af-395a-4580-b9f1-2fc58701971e admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.270888 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa9af-395a-4580-b9f1-2fc58701971e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.271563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa9af-395a-4580-b9f1-2fc58701971e admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.272339 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fa9af-395a-4580-b9f1-2fc58701971e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:23.274777 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fa9af-395a-4580-b9f1-2fc58701971e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:23.285868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:23.318414 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1375/2756] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:03:23 2022] POST /identity/v3/users => generated 281 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:23.322143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa9af-395a-4580-b9f1-2fc58701971e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:23.330015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7979-1b76-40c6-8dfd-afbe35aa14a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:23.331614 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7979-1b76-40c6-8dfd-afbe35aa14a8 service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.331960 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7979-1b76-40c6-8dfd-afbe35aa14a8 service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.332283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7979-1b76-40c6-8dfd-afbe35aa14a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.334205 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-02db7979-1b76-40c6-8dfd-afbe35aa14a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:23.335590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa9af-395a-4580-b9f1-2fc58701971e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:23.340519 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d57a73-78c5-4d8c-9eea-3b7cfc48b7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:23.341344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d57a73-78c5-4d8c-9eea-3b7cfc48b7dc admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.341819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d57a73-78c5-4d8c-9eea-3b7cfc48b7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.342275 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d57a73-78c5-4d8c-9eea-3b7cfc48b7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.343084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d57a73-78c5-4d8c-9eea-3b7cfc48b7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:23.344970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d57a73-78c5-4d8c-9eea-3b7cfc48b7dc admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:23.350005 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db7979-1b76-40c6-8dfd-afbe35aa14a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:23.351827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db7979-1b76-40c6-8dfd-afbe35aa14a8 service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:23.357432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1376/2757] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:23.361761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d57a73-78c5-4d8c-9eea-3b7cfc48b7dc admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:23.373189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d57a73-78c5-4d8c-9eea-3b7cfc48b7dc admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:23.376201 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1382/2758] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:23.379416 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6806e9-fde1-4207-bf49-48b93c720fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:23.380155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6806e9-fde1-4207-bf49-48b93c720fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.380502 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6806e9-fde1-4207-bf49-48b93c720fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.380853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6806e9-fde1-4207-bf49-48b93c720fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.381614 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6806e9-fde1-4207-bf49-48b93c720fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:23.383618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6806e9-fde1-4207-bf49-48b93c720fe3 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:23.395593 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1377/2759] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:23.399466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7979-1b76-40c6-8dfd-afbe35aa14a8 service nova] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:23.415134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7979-1b76-40c6-8dfd-afbe35aa14a8 service nova] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:23.421270 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e73e0-eee8-4553-bec3-c90c4bb6d1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:23.422638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e73e0-eee8-4553-bec3-c90c4bb6d1dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.423016 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e73e0-eee8-4553-bec3-c90c4bb6d1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.424127 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e73e0-eee8-4553-bec3-c90c4bb6d1dc admin admin] PATH_INFO: `/v3/projects/2309fb48a8b243faa61670ff8a277a47/users/c1a3e5ce79354525b5c88ce5a740970c/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.439567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e73e0-eee8-4553-bec3-c90c4bb6d1dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2309fb48a8b243faa61670ff8a277a47, user_id=c1a3e5ce79354525b5c88ce5a740970c, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:23.442477 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e73e0-eee8-4553-bec3-c90c4bb6d1dc admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:23.457401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1383/2760] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:23 2022] PUT /identity/v3/projects/2309fb48a8b243faa61670ff8a277a47/users/c1a3e5ce79354525b5c88ce5a740970c/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:23.468172 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe8a54-2b8a-41eb-975e-656169ffbc95 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.468487 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe8a54-2b8a-41eb-975e-656169ffbc95 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.468764 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe8a54-2b8a-41eb-975e-656169ffbc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.496376 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-f0fe8a54-2b8a-41eb-975e-656169ffbc95 None None] MFA Rules not processed for user `c1a3e5ce79354525b5c88ce5a740970c`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:23.497627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f0fe8a54-2b8a-41eb-975e-656169ffbc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:23.566660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1378/2761] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:23.601865 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e73e0-eee8-4553-bec3-c90c4bb6d1dc admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:23.603928 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9e3e73e0-eee8-4553-bec3-c90c4bb6d1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:23.665280 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e73e0-eee8-4553-bec3-c90c4bb6d1dc admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:23.669083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56991602-1437-4b56-ab73-0b71288d6d53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:23.669736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56991602-1437-4b56-ab73-0b71288d6d53 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.670015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56991602-1437-4b56-ab73-0b71288d6d53 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.670285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56991602-1437-4b56-ab73-0b71288d6d53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.679912 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56991602-1437-4b56-ab73-0b71288d6d53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:23.681371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56991602-1437-4b56-ab73-0b71288d6d53 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:23.690316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1384/2762] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:23.705529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe8a54-2b8a-41eb-975e-656169ffbc95 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:23.769449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe8a54-2b8a-41eb-975e-656169ffbc95 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:23.774047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:23.774677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.774958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.775235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] PATH_INFO: `/v3/users/c1a3e5ce79354525b5c88ce5a740970c` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.776455 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1a3e5ce79354525b5c88ce5a740970c)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:23.778480 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:23.826675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1a3e5ce79354525b5c88ce5a740970c'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:23.827247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1a3e5ce79354525b5c88ce5a740970c'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:23.834637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:23.836932 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1a3e5ce79354525b5c88ce5a740970c'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:23.847358 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1a3e5ce79354525b5c88ce5a740970c'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:23.848286 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1a3e5ce79354525b5c88ce5a740970c'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:23.867924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1379/2763] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:23 2022] DELETE /identity/v3/users/c1a3e5ce79354525b5c88ce5a740970c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:23.872818 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76565b8-ebb5-4d59-9ca8-24e199d33ea2 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:23.873116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76565b8-ebb5-4d59-9ca8-24e199d33ea2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:23.873458 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76565b8-ebb5-4d59-9ca8-24e199d33ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:23.893298 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f76565b8-ebb5-4d59-9ca8-24e199d33ea2 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:23.894536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f76565b8-ebb5-4d59-9ca8-24e199d33ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:23.952601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1385/2764] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:23.963307 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:23.965005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:24.027076 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8a968-7556-481c-aaba-6f007aae1651 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:24.030812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba3d48-97c0-474b-be35-8bc515a126c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:24.031985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba3d48-97c0-474b-be35-8bc515a126c2 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.032262 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba3d48-97c0-474b-be35-8bc515a126c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.032531 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba3d48-97c0-474b-be35-8bc515a126c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:24.042389 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba3d48-97c0-474b-be35-8bc515a126c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:24.043831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba3d48-97c0-474b-be35-8bc515a126c2 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:24.054685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1380/2765] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:24.324158 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76565b8-ebb5-4d59-9ca8-24e199d33ea2 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:24.325698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f76565b8-ebb5-4d59-9ca8-24e199d33ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:24.338545 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76565b8-ebb5-4d59-9ca8-24e199d33ea2 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:24.343830 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:24.344589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.344936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.345273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] PATH_INFO: `/v3/projects/2309fb48a8b243faa61670ff8a277a47` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:24.346491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2309fb48a8b243faa61670ff8a277a47)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:24.348579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:24.386910 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:24.398232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2309fb48a8b243faa61670ff8a277a47', 'request_id': 'req-0fbb43c8-a7ea-4417-9822-544197123601'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:24.405072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2309fb48a8b243faa61670ff8a277a47', 'request_id': 'req-0fbb43c8-a7ea-4417-9822-544197123601'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:24.411305 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2309fb48a8b243faa61670ff8a277a47', 'request_id': 'req-0fbb43c8-a7ea-4417-9822-544197123601'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:24.421427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] The token cache is being invalidate because project 2309fb48a8b243faa61670ff8a277a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:24.421756 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:24.422373 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-0fbb43c8-a7ea-4417-9822-544197123601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:24.423821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1386/2766] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:24 2022] DELETE /identity/v3/projects/2309fb48a8b243faa61670ff8a277a47 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:24.428620 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d125c18-cab2-48ac-a8d5-cf43bb8b0506 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.428897 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d125c18-cab2-48ac-a8d5-cf43bb8b0506 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.429166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d125c18-cab2-48ac-a8d5-cf43bb8b0506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:24.448557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-8d125c18-cab2-48ac-a8d5-cf43bb8b0506 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:24.449767 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8d125c18-cab2-48ac-a8d5-cf43bb8b0506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:24.511094 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1381/2767] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:24.520040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec202a-0981-4aa0-8ffd-6630c6f28a27 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.520361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec202a-0981-4aa0-8ffd-6630c6f28a27 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.520677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec202a-0981-4aa0-8ffd-6630c6f28a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:24.540489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-03ec202a-0981-4aa0-8ffd-6630c6f28a27 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:24.541743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-03ec202a-0981-4aa0-8ffd-6630c6f28a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:24.559741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1387/2768] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:24.561834 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d125c18-cab2-48ac-a8d5-cf43bb8b0506 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:24.580101 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d125c18-cab2-48ac-a8d5-cf43bb8b0506 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:24.585940 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8af7b-639d-4dc2-b8ab-10e2d7d9c1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:24.586630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8af7b-639d-4dc2-b8ab-10e2d7d9c1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.586906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8af7b-639d-4dc2-b8ab-10e2d7d9c1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.587178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8af7b-639d-4dc2-b8ab-10e2d7d9c1d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:24.587823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8af7b-639d-4dc2-b8ab-10e2d7d9c1d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:03:24.588123 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8af7b-639d-4dc2-b8ab-10e2d7d9c1d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:24.589893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8af7b-639d-4dc2-b8ab-10e2d7d9c1d1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:24.604312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1382/2769] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:03:24.608282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec202a-0981-4aa0-8ffd-6630c6f28a27 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:24.620022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec202a-0981-4aa0-8ffd-6630c6f28a27 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:24.626017 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effab78e-633c-4d00-a1f8-644ea4d786cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:24.626911 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effab78e-633c-4d00-a1f8-644ea4d786cb admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.627282 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effab78e-633c-4d00-a1f8-644ea4d786cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.627665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effab78e-633c-4d00-a1f8-644ea4d786cb admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:24.628384 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effab78e-633c-4d00-a1f8-644ea4d786cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:24.630597 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effab78e-633c-4d00-a1f8-644ea4d786cb admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:24.685543 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1388/2770] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:03:24 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:03:24.688095 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8af7b-639d-4dc2-b8ab-10e2d7d9c1d1 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:24.698030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8af7b-639d-4dc2-b8ab-10e2d7d9c1d1 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:24.756636 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8e550-f983-45db-9796-9ef5b3f429f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:24.757239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8e550-f983-45db-9796-9ef5b3f429f9 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.757536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8e550-f983-45db-9796-9ef5b3f429f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.757808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8e550-f983-45db-9796-9ef5b3f429f9 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:24.758449 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8e550-f983-45db-9796-9ef5b3f429f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:24.760353 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8e550-f983-45db-9796-9ef5b3f429f9 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:24.769122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:24.783808 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1383/2771] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:03:24 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:03:24.787823 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effab78e-633c-4d00-a1f8-644ea4d786cb admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:24.798441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effab78e-633c-4d00-a1f8-644ea4d786cb admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:24.803290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3f95f-c5b8-4d7b-ad26-9c1a54539815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:24.803968 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3f95f-c5b8-4d7b-ad26-9c1a54539815 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.804311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3f95f-c5b8-4d7b-ad26-9c1a54539815 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.804662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3f95f-c5b8-4d7b-ad26-9c1a54539815 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:24.805356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3f95f-c5b8-4d7b-ad26-9c1a54539815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:24.806990 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3f95f-c5b8-4d7b-ad26-9c1a54539815 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:24.818650 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1389/2772] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:03:24.822635 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8e550-f983-45db-9796-9ef5b3f429f9 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:24.833091 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8e550-f983-45db-9796-9ef5b3f429f9 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:24.837802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e771e-5a4e-48ae-8f02-54b711aff01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:24.838475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e771e-5a4e-48ae-8f02-54b711aff01c admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.838753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e771e-5a4e-48ae-8f02-54b711aff01c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.839016 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e771e-5a4e-48ae-8f02-54b711aff01c admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:24.839650 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e771e-5a4e-48ae-8f02-54b711aff01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:24.841235 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e771e-5a4e-48ae-8f02-54b711aff01c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:24.852031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1384/2773] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:03:24.856416 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3f95f-c5b8-4d7b-ad26-9c1a54539815 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:24.867645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3f95f-c5b8-4d7b-ad26-9c1a54539815 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:24.872943 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0384d-2795-40ce-9091-8228f4ed3653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:24.873634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0384d-2795-40ce-9091-8228f4ed3653 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.873915 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0384d-2795-40ce-9091-8228f4ed3653 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.874187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0384d-2795-40ce-9091-8228f4ed3653 admin admin] PATH_INFO: `/v3/projects/337057fdbed641899c7a0e33a56f419d/users/590389e9e58e4dfd925ba7633fdce40f/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:24.889686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0384d-2795-40ce-9091-8228f4ed3653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=337057fdbed641899c7a0e33a56f419d, user_id=590389e9e58e4dfd925ba7633fdce40f, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:24.892388 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0384d-2795-40ce-9091-8228f4ed3653 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:24.907622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1390/2774] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:24 2022] PUT /identity/v3/projects/337057fdbed641899c7a0e33a56f419d/users/590389e9e58e4dfd925ba7633fdce40f/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:24.975194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b479-9f46-457c-98ce-5ca2fdc11c65 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:24.975746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b479-9f46-457c-98ce-5ca2fdc11c65 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:24.975746 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b479-9f46-457c-98ce-5ca2fdc11c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:25.004304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-6239b479-9f46-457c-98ce-5ca2fdc11c65 None None] MFA Rules not processed for user `590389e9e58e4dfd925ba7633fdce40f`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:25.005578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6239b479-9f46-457c-98ce-5ca2fdc11c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:25.069944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1385/2775] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:25.080551 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0384d-2795-40ce-9091-8228f4ed3653 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:25.082605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-24e0384d-2795-40ce-9091-8228f4ed3653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:25.138263 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0384d-2795-40ce-9091-8228f4ed3653 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:25.141975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7924f6-c0a7-4f9f-9ca5-9fb2f16a4bac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:25.142521 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7924f6-c0a7-4f9f-9ca5-9fb2f16a4bac service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:25.142801 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7924f6-c0a7-4f9f-9ca5-9fb2f16a4bac service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:25.143079 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7924f6-c0a7-4f9f-9ca5-9fb2f16a4bac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:25.151795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7924f6-c0a7-4f9f-9ca5-9fb2f16a4bac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:25.153676 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7924f6-c0a7-4f9f-9ca5-9fb2f16a4bac service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:25.164609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1391/2776] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:29.279731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239b479-9f46-457c-98ce-5ca2fdc11c65 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:29.342613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239b479-9f46-457c-98ce-5ca2fdc11c65 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:29.348116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:29.348743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:29.349057 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:29.349325 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] PATH_INFO: `/v3/users/590389e9e58e4dfd925ba7633fdce40f` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:29.350461 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=590389e9e58e4dfd925ba7633fdce40f)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:29.352492 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:29.400886 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590389e9e58e4dfd925ba7633fdce40f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:29.401448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590389e9e58e4dfd925ba7633fdce40f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:29.411770 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590389e9e58e4dfd925ba7633fdce40f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:29.420903 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590389e9e58e4dfd925ba7633fdce40f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:29.421797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590389e9e58e4dfd925ba7633fdce40f'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:29.431543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:29.445230 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1386/2777] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:29 2022] DELETE /identity/v3/users/590389e9e58e4dfd925ba7633fdce40f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:29.449790 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f6859-0a4e-4dbe-8438-e12529d36ee5 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:29.450301 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f6859-0a4e-4dbe-8438-e12529d36ee5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:29.450441 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f6859-0a4e-4dbe-8438-e12529d36ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:29.470405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f78f6859-0a4e-4dbe-8438-e12529d36ee5 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:29.471609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f78f6859-0a4e-4dbe-8438-e12529d36ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:29.528253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1392/2778] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:29.537977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:29.539350 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:29.597517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06362fe9-6f47-4b92-aece-80fba4f4b071 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:29.601258 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a8107-751c-4d83-91ed-c722acab87b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:29.601885 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a8107-751c-4d83-91ed-c722acab87b4 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:29.602157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a8107-751c-4d83-91ed-c722acab87b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:29.602423 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a8107-751c-4d83-91ed-c722acab87b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:29.611257 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a8107-751c-4d83-91ed-c722acab87b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:29.612679 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a8107-751c-4d83-91ed-c722acab87b4 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:29.622028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1387/2779] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:29.894826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f6859-0a4e-4dbe-8438-e12529d36ee5 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:29.896240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f78f6859-0a4e-4dbe-8438-e12529d36ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:29.914210 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:29.920817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f6859-0a4e-4dbe-8438-e12529d36ee5 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:29.929048 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:29.929846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:29.930504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:29.931106 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] PATH_INFO: `/v3/projects/337057fdbed641899c7a0e33a56f419d` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:29.932630 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=337057fdbed641899c7a0e33a56f419d)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:29.934675 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:29.980312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '337057fdbed641899c7a0e33a56f419d', 'request_id': 'req-4021e317-4a02-4f9c-9081-19b68c24b527'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:29.987508 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '337057fdbed641899c7a0e33a56f419d', 'request_id': 'req-4021e317-4a02-4f9c-9081-19b68c24b527'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:29.993587 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '337057fdbed641899c7a0e33a56f419d', 'request_id': 'req-4021e317-4a02-4f9c-9081-19b68c24b527'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:30.004214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] The token cache is being invalidate because project 337057fdbed641899c7a0e33a56f419d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:30.004499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:30.005126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:30.006611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1393/2780] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:29 2022] DELETE /identity/v3/projects/337057fdbed641899c7a0e33a56f419d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:33.090561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a8107-751c-4d83-91ed-c722acab87b4 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:33.093894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-c07a8107-751c-4d83-91ed-c722acab87b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:33.177452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a8107-751c-4d83-91ed-c722acab87b4 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:33.241812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9716efe-7fa9-4570-9bb6-f337cf530de8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:33.242445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9716efe-7fa9-4570-9bb6-f337cf530de8 service nova] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:33.242726 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9716efe-7fa9-4570-9bb6-f337cf530de8 service nova] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:33.243038 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9716efe-7fa9-4570-9bb6-f337cf530de8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:33.244868 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e9716efe-7fa9-4570-9bb6-f337cf530de8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:33.257597 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9716efe-7fa9-4570-9bb6-f337cf530de8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:33.259213 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9716efe-7fa9-4570-9bb6-f337cf530de8 service nova] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:33.270304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1388/2781] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:36.552140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:36.553686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:36.614214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021e317-4a02-4f9c-9081-19b68c24b527 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:36.618224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2189963-8d8d-4714-820c-96408b27bb29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:36.619040 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2189963-8d8d-4714-820c-96408b27bb29 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:36.619434 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2189963-8d8d-4714-820c-96408b27bb29 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:36.619716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2189963-8d8d-4714-820c-96408b27bb29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:36.621586 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-b2189963-8d8d-4714-820c-96408b27bb29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:36.691788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2189963-8d8d-4714-820c-96408b27bb29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:36.693229 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2189963-8d8d-4714-820c-96408b27bb29 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:36.702746 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1394/2782] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:03:37.698792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9716efe-7fa9-4570-9bb6-f337cf530de8 service nova] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:37.700201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-e9716efe-7fa9-4570-9bb6-f337cf530de8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:37.713674 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9716efe-7fa9-4570-9bb6-f337cf530de8 service nova] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:37.718851 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:37.719469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:37.720238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:37.720507 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] PATH_INFO: `/v3/users/bbd76f97aae64c2cad891a46951592af` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:37.721651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd76f97aae64c2cad891a46951592af)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:37.723554 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:37.735392 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:37.774454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd76f97aae64c2cad891a46951592af'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:37.775044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd76f97aae64c2cad891a46951592af'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:37.782064 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd76f97aae64c2cad891a46951592af'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:37.789681 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd76f97aae64c2cad891a46951592af'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:37.790417 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd76f97aae64c2cad891a46951592af'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:37.808865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1389/2783] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:37 2022] DELETE /identity/v3/users/bbd76f97aae64c2cad891a46951592af => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:37.813253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa46ad-ce0e-419a-984a-2cf085047842 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:37.813826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa46ad-ce0e-419a-984a-2cf085047842 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:37.813936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa46ad-ce0e-419a-984a-2cf085047842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:37.837228 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-3eaa46ad-ce0e-419a-984a-2cf085047842 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:37.838541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3eaa46ad-ce0e-419a-984a-2cf085047842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:37.875214 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:37.905780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1395/2784] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:03:37.916108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:37.917643 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:37.983958 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cdd91-60e7-4315-a0b0-0c00f4be2725 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:37.988452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a402f71-4893-4343-8ffb-27fdcf171253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:37.989077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a402f71-4893-4343-8ffb-27fdcf171253 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:37.989359 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a402f71-4893-4343-8ffb-27fdcf171253 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:37.989673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a402f71-4893-4343-8ffb-27fdcf171253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.000150 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a402f71-4893-4343-8ffb-27fdcf171253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:38.003015 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a402f71-4893-4343-8ffb-27fdcf171253 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:38.015468 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1390/2785] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:38.309076 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa46ad-ce0e-419a-984a-2cf085047842 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:38.310533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3eaa46ad-ce0e-419a-984a-2cf085047842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:38.323547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa46ad-ce0e-419a-984a-2cf085047842 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:38.328356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:38.328999 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:38.329843 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:38.330119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] PATH_INFO: `/v3/projects/4b70f657e3d646edb71857f40926146d` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.334442 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b70f657e3d646edb71857f40926146d)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:38.336601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:38.402873 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b70f657e3d646edb71857f40926146d', 'request_id': 'req-b82878e5-344d-4eef-a849-7258b09c4c95'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:38.413255 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b70f657e3d646edb71857f40926146d', 'request_id': 'req-b82878e5-344d-4eef-a849-7258b09c4c95'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:38.420810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b70f657e3d646edb71857f40926146d', 'request_id': 'req-b82878e5-344d-4eef-a849-7258b09c4c95'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:38.433072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] The token cache is being invalidate because project 4b70f657e3d646edb71857f40926146d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:38.433363 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:38.434139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-b82878e5-344d-4eef-a849-7258b09c4c95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:38.436045 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1396/2786] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:38 2022] DELETE /identity/v3/projects/4b70f657e3d646edb71857f40926146d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:38.440662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44808ec5-d391-4a00-bc6b-dc34a6f69b6c None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:38.441018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44808ec5-d391-4a00-bc6b-dc34a6f69b6c None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:38.441354 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44808ec5-d391-4a00-bc6b-dc34a6f69b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.462722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-44808ec5-d391-4a00-bc6b-dc34a6f69b6c None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:38.464421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-44808ec5-d391-4a00-bc6b-dc34a6f69b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:38.526673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1391/2787] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:03:38.534575 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffb352-3dab-4d46-bc84-7fba7ae10525 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:38.534950 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffb352-3dab-4d46-bc84-7fba7ae10525 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:38.535299 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffb352-3dab-4d46-bc84-7fba7ae10525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.555880 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-18ffb352-3dab-4d46-bc84-7fba7ae10525 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:38.557203 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-18ffb352-3dab-4d46-bc84-7fba7ae10525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:38.566129 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1397/2788] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:38.570506 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44808ec5-d391-4a00-bc6b-dc34a6f69b6c None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:38.581489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44808ec5-d391-4a00-bc6b-dc34a6f69b6c None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:38.586508 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f84b7-29d0-45fb-8639-3500b125aa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:38.587174 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f84b7-29d0-45fb-8639-3500b125aa67 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:38.587458 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f84b7-29d0-45fb-8639-3500b125aa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:38.587730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f84b7-29d0-45fb-8639-3500b125aa67 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.588382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f84b7-29d0-45fb-8639-3500b125aa67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:03:38.588675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f84b7-29d0-45fb-8639-3500b125aa67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:38.590418 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f84b7-29d0-45fb-8639-3500b125aa67 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:38.607523 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1392/2789] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:03:38.608860 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffb352-3dab-4d46-bc84-7fba7ae10525 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:38.619397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffb352-3dab-4d46-bc84-7fba7ae10525 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:38.625075 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb272833-379b-4bc3-870a-b9609e8b322d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:38.626405 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb272833-379b-4bc3-870a-b9609e8b322d admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:38.626778 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb272833-379b-4bc3-870a-b9609e8b322d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:38.627149 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb272833-379b-4bc3-870a-b9609e8b322d admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.627914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb272833-379b-4bc3-870a-b9609e8b322d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:38.629977 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb272833-379b-4bc3-870a-b9609e8b322d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:38.665191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1398/2790] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:03:38 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:03:38.666680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f84b7-29d0-45fb-8639-3500b125aa67 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:38.677459 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f84b7-29d0-45fb-8639-3500b125aa67 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:38.692787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:38.746314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789299e7-36e2-47b7-8d92-892e04445772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:38.747075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789299e7-36e2-47b7-8d92-892e04445772 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:38.747451 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789299e7-36e2-47b7-8d92-892e04445772 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:38.747817 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789299e7-36e2-47b7-8d92-892e04445772 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.748563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789299e7-36e2-47b7-8d92-892e04445772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:38.750727 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789299e7-36e2-47b7-8d92-892e04445772 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:38.777831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1393/2791] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:03:38 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:38.779481 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb272833-379b-4bc3-870a-b9609e8b322d admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:38.791241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb272833-379b-4bc3-870a-b9609e8b322d admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:38.796460 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b7cfb-b2a0-405a-8e7e-88addc0cfe19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:38.797252 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b7cfb-b2a0-405a-8e7e-88addc0cfe19 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:38.797670 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b7cfb-b2a0-405a-8e7e-88addc0cfe19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:38.798077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b7cfb-b2a0-405a-8e7e-88addc0cfe19 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.798825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b7cfb-b2a0-405a-8e7e-88addc0cfe19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:38.800760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b7cfb-b2a0-405a-8e7e-88addc0cfe19 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:38.815001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1399/2792] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:38.816239 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789299e7-36e2-47b7-8d92-892e04445772 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:38.827899 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789299e7-36e2-47b7-8d92-892e04445772 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:38.834758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a6274-7fc7-4872-8c47-8743ea8b3644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:38.835408 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a6274-7fc7-4872-8c47-8743ea8b3644 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:38.835708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a6274-7fc7-4872-8c47-8743ea8b3644 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:38.835970 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a6274-7fc7-4872-8c47-8743ea8b3644 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.836618 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a6274-7fc7-4872-8c47-8743ea8b3644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:38.838260 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a6274-7fc7-4872-8c47-8743ea8b3644 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:38.850018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1394/2793] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:03:38.854245 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b7cfb-b2a0-405a-8e7e-88addc0cfe19 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:38.866304 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b7cfb-b2a0-405a-8e7e-88addc0cfe19 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:38.871391 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529b2bc-de98-4092-9e2b-be9cf958e0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:38.872369 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529b2bc-de98-4092-9e2b-be9cf958e0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:38.872745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529b2bc-de98-4092-9e2b-be9cf958e0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:38.873115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529b2bc-de98-4092-9e2b-be9cf958e0f1 admin admin] PATH_INFO: `/v3/projects/45fddbb738184ecfa8bfb9ea51f61b58/users/930d4934ebe64451a59d6a96ed5d2099/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.889105 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529b2bc-de98-4092-9e2b-be9cf958e0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fddbb738184ecfa8bfb9ea51f61b58, user_id=930d4934ebe64451a59d6a96ed5d2099, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:38.891965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529b2bc-de98-4092-9e2b-be9cf958e0f1 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:38.908558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1400/2794] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:38 2022] PUT /identity/v3/projects/45fddbb738184ecfa8bfb9ea51f61b58/users/930d4934ebe64451a59d6a96ed5d2099/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:38.918967 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3387f0-42ec-4909-8cd2-2412899cb359 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:38.919256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3387f0-42ec-4909-8cd2-2412899cb359 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:38.919558 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3387f0-42ec-4909-8cd2-2412899cb359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:38.949975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-5a3387f0-42ec-4909-8cd2-2412899cb359 None None] MFA Rules not processed for user `930d4934ebe64451a59d6a96ed5d2099`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:38.951396 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-5a3387f0-42ec-4909-8cd2-2412899cb359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:39.028293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1395/2795] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:39.033356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529b2bc-de98-4092-9e2b-be9cf958e0f1 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.101620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:39.105766 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529b2bc-de98-4092-9e2b-be9cf958e0f1 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.112320 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0c16-440c-44e2-8ac0-9a0721481dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.113009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0c16-440c-44e2-8ac0-9a0721481dbc admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.113563 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0c16-440c-44e2-8ac0-9a0721481dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.114059 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0c16-440c-44e2-8ac0-9a0721481dbc admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.114958 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0c16-440c-44e2-8ac0-9a0721481dbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.116864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0c16-440c-44e2-8ac0-9a0721481dbc admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:39.143988 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1401/2796] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:03:39 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:39.147976 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3387f0-42ec-4909-8cd2-2412899cb359 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.159540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3387f0-42ec-4909-8cd2-2412899cb359 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.224863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae2209-8317-40a6-8202-5e2f75a6f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.224863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae2209-8317-40a6-8202-5e2f75a6f8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.224863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae2209-8317-40a6-8202-5e2f75a6f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.224863 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae2209-8317-40a6-8202-5e2f75a6f8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.225397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae2209-8317-40a6-8202-5e2f75a6f8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.225397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae2209-8317-40a6-8202-5e2f75a6f8c4 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:39.246457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1396/2797] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:03:39 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:39.250547 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0c16-440c-44e2-8ac0-9a0721481dbc admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.263349 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0c16-440c-44e2-8ac0-9a0721481dbc admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.268252 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906a364-a1f9-4150-98a7-3a4e77add59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.268888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a364-a1f9-4150-98a7-3a4e77add59d admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.269171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a364-a1f9-4150-98a7-3a4e77add59d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.269448 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a364-a1f9-4150-98a7-3a4e77add59d admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.270152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906a364-a1f9-4150-98a7-3a4e77add59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.271871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906a364-a1f9-4150-98a7-3a4e77add59d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:39.283400 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1402/2798] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:39.287638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae2209-8317-40a6-8202-5e2f75a6f8c4 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.298821 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae2209-8317-40a6-8202-5e2f75a6f8c4 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.303421 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b06023-3eb0-4057-8a9e-6fca9f87d085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.304021 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06023-3eb0-4057-8a9e-6fca9f87d085 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.304300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06023-3eb0-4057-8a9e-6fca9f87d085 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.304574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06023-3eb0-4057-8a9e-6fca9f87d085 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.305211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b06023-3eb0-4057-8a9e-6fca9f87d085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.306872 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b06023-3eb0-4057-8a9e-6fca9f87d085 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:39.320395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1397/2799] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:39.321892 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906a364-a1f9-4150-98a7-3a4e77add59d admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.335836 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906a364-a1f9-4150-98a7-3a4e77add59d admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.345777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc63f2-81df-4d3a-a39b-6cfdefbf4646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.346453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc63f2-81df-4d3a-a39b-6cfdefbf4646 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.346730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc63f2-81df-4d3a-a39b-6cfdefbf4646 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.347000 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc63f2-81df-4d3a-a39b-6cfdefbf4646 admin admin] PATH_INFO: `/v3/projects/f491af6306ff4747b0d6347c0495c39a/users/74acfa4c97d2416e908b9dbcbbd08f06/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.363070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc63f2-81df-4d3a-a39b-6cfdefbf4646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f491af6306ff4747b0d6347c0495c39a, user_id=74acfa4c97d2416e908b9dbcbbd08f06, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.365669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc63f2-81df-4d3a-a39b-6cfdefbf4646 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:39.385658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1403/2800] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:39 2022] PUT /identity/v3/projects/f491af6306ff4747b0d6347c0495c39a/users/74acfa4c97d2416e908b9dbcbbd08f06/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:39.395908 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b5053-d673-47e8-a1fb-ce9e2ad2d979 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.396276 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b5053-d673-47e8-a1fb-ce9e2ad2d979 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.396621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b5053-d673-47e8-a1fb-ce9e2ad2d979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.405765 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:39.441612 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-621b5053-d673-47e8-a1fb-ce9e2ad2d979 None None] MFA Rules not processed for user `74acfa4c97d2416e908b9dbcbbd08f06`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:39.442846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-621b5053-d673-47e8-a1fb-ce9e2ad2d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:39.530633 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1398/2801] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:39.535499 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc63f2-81df-4d3a-a39b-6cfdefbf4646 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.600064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc63f2-81df-4d3a-a39b-6cfdefbf4646 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.605343 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb1bc7-90c3-460b-8ca2-8984b73093ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.605991 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb1bc7-90c3-460b-8ca2-8984b73093ca admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.606258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb1bc7-90c3-460b-8ca2-8984b73093ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.606537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb1bc7-90c3-460b-8ca2-8984b73093ca admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.607193 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb1bc7-90c3-460b-8ca2-8984b73093ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.609241 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb1bc7-90c3-460b-8ca2-8984b73093ca admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:39.639556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1404/2802] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:03:39 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:39.644245 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b5053-d673-47e8-a1fb-ce9e2ad2d979 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.656355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b5053-d673-47e8-a1fb-ce9e2ad2d979 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.743852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd19637-f815-40ef-990c-39fbd189f416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.744605 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd19637-f815-40ef-990c-39fbd189f416 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.744938 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd19637-f815-40ef-990c-39fbd189f416 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.745155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd19637-f815-40ef-990c-39fbd189f416 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.745842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd19637-f815-40ef-990c-39fbd189f416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.747875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd19637-f815-40ef-990c-39fbd189f416 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:39.776446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1399/2803] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:03:39 2022] POST /identity/v3/users => generated 298 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:39.778010 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb1bc7-90c3-460b-8ca2-8984b73093ca admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.790093 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb1bc7-90c3-460b-8ca2-8984b73093ca admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.795513 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef4591-2e9c-49b2-9df3-1c3cfb12b1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.796140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef4591-2e9c-49b2-9df3-1c3cfb12b1f2 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.796424 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef4591-2e9c-49b2-9df3-1c3cfb12b1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.796702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef4591-2e9c-49b2-9df3-1c3cfb12b1f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.797367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef4591-2e9c-49b2-9df3-1c3cfb12b1f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.799058 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef4591-2e9c-49b2-9df3-1c3cfb12b1f2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:39.810646 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1405/2804] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:39.814892 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd19637-f815-40ef-990c-39fbd189f416 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.825130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd19637-f815-40ef-990c-39fbd189f416 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.837460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6261f7-700f-44f2-b1ca-f79afa8e6096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.838155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6261f7-700f-44f2-b1ca-f79afa8e6096 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.838431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6261f7-700f-44f2-b1ca-f79afa8e6096 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.838702 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6261f7-700f-44f2-b1ca-f79afa8e6096 admin admin] PATH_INFO: `/v3/projects/99df29a35cff47b9b27914dfe5ce9863/users/32c4d5ed7a474411bdc9cf0daf35a4eb/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.855355 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6261f7-700f-44f2-b1ca-f79afa8e6096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99df29a35cff47b9b27914dfe5ce9863, user_id=32c4d5ed7a474411bdc9cf0daf35a4eb, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.858029 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6261f7-700f-44f2-b1ca-f79afa8e6096 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:39.872054 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1400/2805] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:39 2022] PUT /identity/v3/projects/99df29a35cff47b9b27914dfe5ce9863/users/32c4d5ed7a474411bdc9cf0daf35a4eb/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:39.876187 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef4591-2e9c-49b2-9df3-1c3cfb12b1f2 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.886036 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef4591-2e9c-49b2-9df3-1c3cfb12b1f2 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.946798 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7ade8-f81f-47ec-92a8-edf529e14370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.947407 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7ade8-f81f-47ec-92a8-edf529e14370 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.947688 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7ade8-f81f-47ec-92a8-edf529e14370 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.947964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7ade8-f81f-47ec-92a8-edf529e14370 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.948613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7ade8-f81f-47ec-92a8-edf529e14370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.950239 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7ade8-f81f-47ec-92a8-edf529e14370 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:39.960729 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1406/2806] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:03:39.964994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6261f7-700f-44f2-b1ca-f79afa8e6096 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:39.976046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6261f7-700f-44f2-b1ca-f79afa8e6096 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:39.980680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ad770-96b7-476a-92de-6347e63fb374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:39.981334 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ad770-96b7-476a-92de-6347e63fb374 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:39.981654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ad770-96b7-476a-92de-6347e63fb374 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:39.981928 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ad770-96b7-476a-92de-6347e63fb374 admin admin] PATH_INFO: `/v3/projects/99df29a35cff47b9b27914dfe5ce9863/users/32c4d5ed7a474411bdc9cf0daf35a4eb/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:39.983835 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ad770-96b7-476a-92de-6347e63fb374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99df29a35cff47b9b27914dfe5ce9863, user_id=32c4d5ed7a474411bdc9cf0daf35a4eb, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:39.989476 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ad770-96b7-476a-92de-6347e63fb374 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:40.010036 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1401/2807] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:39 2022] PUT /identity/v3/projects/99df29a35cff47b9b27914dfe5ce9863/users/32c4d5ed7a474411bdc9cf0daf35a4eb/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:40.011486 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b972fdf-9e26-4b78-b75b-1e5fdc4cc038 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:40.011787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b972fdf-9e26-4b78-b75b-1e5fdc4cc038 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:40.012101 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b972fdf-9e26-4b78-b75b-1e5fdc4cc038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:40.043715 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6b972fdf-9e26-4b78-b75b-1e5fdc4cc038 None None] MFA Rules not processed for user `32c4d5ed7a474411bdc9cf0daf35a4eb`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:40.044929 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6b972fdf-9e26-4b78-b75b-1e5fdc4cc038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:40.050546 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:40.138353 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1407/2808] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:03:40.149993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ad770-96b7-476a-92de-6347e63fb374 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:40.153867 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-668ad770-96b7-476a-92de-6347e63fb374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:40.228321 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:40.236551 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ad770-96b7-476a-92de-6347e63fb374 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:40.242706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e3e53-59fc-4e28-a871-890a3ad00b34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:40.243690 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e3e53-59fc-4e28-a871-890a3ad00b34 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:40.243973 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e3e53-59fc-4e28-a871-890a3ad00b34 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:40.244247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e3e53-59fc-4e28-a871-890a3ad00b34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:40.310513 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e3e53-59fc-4e28-a871-890a3ad00b34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:40.312001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e3e53-59fc-4e28-a871-890a3ad00b34 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:40.321828 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1402/2809] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:43.433402 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b972fdf-9e26-4b78-b75b-1e5fdc4cc038 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:43.434901 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6b972fdf-9e26-4b78-b75b-1e5fdc4cc038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:43.499872 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b972fdf-9e26-4b78-b75b-1e5fdc4cc038 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:43.504997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:43.505659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:43.505945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:43.506221 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] PATH_INFO: `/v3/users/7cfae014351641d8a949775875a45958` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:43.507337 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cfae014351641d8a949775875a45958)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:43.509252 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:43.553029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cfae014351641d8a949775875a45958'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:43.553634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cfae014351641d8a949775875a45958'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:43.560387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cfae014351641d8a949775875a45958'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:43.572453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cfae014351641d8a949775875a45958'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:43.578877 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cfae014351641d8a949775875a45958'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:43.585705 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1408/2810] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:43 2022] DELETE /identity/v3/users/7cfae014351641d8a949775875a45958 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:43.754842 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e3e53-59fc-4e28-a871-890a3ad00b34 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:43.766999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e3e53-59fc-4e28-a871-890a3ad00b34 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:43.828300 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:43.828914 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:43.829211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:43.829475 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] PATH_INFO: `/v3/projects/a167b2ccacbc4e6d903b09d9a00650fe` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:43.830613 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a167b2ccacbc4e6d903b09d9a00650fe)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:43.832737 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:43.877793 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a167b2ccacbc4e6d903b09d9a00650fe', 'request_id': 'req-4716c78b-8f13-4018-bf5f-dd863bd22bfa'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:43.885208 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a167b2ccacbc4e6d903b09d9a00650fe', 'request_id': 'req-4716c78b-8f13-4018-bf5f-dd863bd22bfa'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:43.891673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a167b2ccacbc4e6d903b09d9a00650fe', 'request_id': 'req-4716c78b-8f13-4018-bf5f-dd863bd22bfa'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:43.902013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] The token cache is being invalidate because project a167b2ccacbc4e6d903b09d9a00650fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:43.902297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:43.902912 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:43.904349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1403/2811] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:43 2022] DELETE /identity/v3/projects/a167b2ccacbc4e6d903b09d9a00650fe => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:43.909001 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:43.910439 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:43.924057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58526e0b-6f7a-4f66-b346-062f011854ea admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:43.942262 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:43.986761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:43.987387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:43.987667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:43.987935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] PATH_INFO: `/v3/users/c480a3ee842149a89c5309598fea23d3` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:43.989052 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c480a3ee842149a89c5309598fea23d3)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:43.990983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:44.035410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c480a3ee842149a89c5309598fea23d3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:44.035978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c480a3ee842149a89c5309598fea23d3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:44.043120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c480a3ee842149a89c5309598fea23d3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:44.055558 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c480a3ee842149a89c5309598fea23d3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:44.062991 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c480a3ee842149a89c5309598fea23d3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:44.069072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1409/2812] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:43 2022] DELETE /identity/v3/users/c480a3ee842149a89c5309598fea23d3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:44.245083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:44.247012 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:44.260000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716c78b-8f13-4018-bf5f-dd863bd22bfa admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:44.289831 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:44.319709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:44.320340 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:44.320621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:44.320894 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] PATH_INFO: `/v3/projects/b761452ca4a9445f802ec1f2e22513c4` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:44.322041 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] RBAC: Authorizing `identity:delete_project(project_id=b761452ca4a9445f802ec1f2e22513c4)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:44.323962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:44.370272 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b761452ca4a9445f802ec1f2e22513c4', 'request_id': 'req-dd91d63d-af47-4791-a391-c3113d230aad'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:44.377757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b761452ca4a9445f802ec1f2e22513c4', 'request_id': 'req-dd91d63d-af47-4791-a391-c3113d230aad'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:44.383749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b761452ca4a9445f802ec1f2e22513c4', 'request_id': 'req-dd91d63d-af47-4791-a391-c3113d230aad'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:44.394720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] The token cache is being invalidate because project b761452ca4a9445f802ec1f2e22513c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:44.394998 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:44.395675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:44.397201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1404/2813] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:44 2022] DELETE /identity/v3/projects/b761452ca4a9445f802ec1f2e22513c4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:45.015965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:45.017659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:45.073086 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a2068-99dd-4787-8180-58ea1f3f9ad8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:45.119619 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b72d7-f5be-47b2-94d0-430b427ef035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:45.120216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b72d7-f5be-47b2-94d0-430b427ef035 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:45.120493 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b72d7-f5be-47b2-94d0-430b427ef035 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:45.120760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b72d7-f5be-47b2-94d0-430b427ef035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:45.122488 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9a6b72d7-f5be-47b2-94d0-430b427ef035 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:45.132691 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:45.181804 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b72d7-f5be-47b2-94d0-430b427ef035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:45.183265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b72d7-f5be-47b2-94d0-430b427ef035 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:45.192494 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1410/2814] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:45.726190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:45.727649 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:45.739621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91d63d-af47-4791-a391-c3113d230aad admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:45.743332 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88195805-7153-44d5-a510-6a0202e1c4d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:45.743955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195805-7153-44d5-a510-6a0202e1c4d6 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:45.744236 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195805-7153-44d5-a510-6a0202e1c4d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:45.744742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195805-7153-44d5-a510-6a0202e1c4d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:45.746562 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-88195805-7153-44d5-a510-6a0202e1c4d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:45.809595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88195805-7153-44d5-a510-6a0202e1c4d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:45.811033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88195805-7153-44d5-a510-6a0202e1c4d6 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:45.820387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1405/2815] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:53.920833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b72d7-f5be-47b2-94d0-430b427ef035 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:53.922327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-9a6b72d7-f5be-47b2-94d0-430b427ef035 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:53.987852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b72d7-f5be-47b2-94d0-430b427ef035 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:53.992325 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:53.992946 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:53.993224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:53.993514 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] PATH_INFO: `/v3/users/930d4934ebe64451a59d6a96ed5d2099` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:53.994616 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=930d4934ebe64451a59d6a96ed5d2099)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:53.996626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:54.040800 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '930d4934ebe64451a59d6a96ed5d2099'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.041367 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '930d4934ebe64451a59d6a96ed5d2099'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.048095 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '930d4934ebe64451a59d6a96ed5d2099'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.059541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '930d4934ebe64451a59d6a96ed5d2099'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.066851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '930d4934ebe64451a59d6a96ed5d2099'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.073134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1411/2816] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:53 2022] DELETE /identity/v3/users/930d4934ebe64451a59d6a96ed5d2099 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:54.077579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159c833-aaee-4c57-abbf-31a90e73808f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:54.078044 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159c833-aaee-4c57-abbf-31a90e73808f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:54.078166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159c833-aaee-4c57-abbf-31a90e73808f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:54.097990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-b159c833-aaee-4c57-abbf-31a90e73808f None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:54.099233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b159c833-aaee-4c57-abbf-31a90e73808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:54.148730 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:54.164471 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1406/2817] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:54.174709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:54.176108 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:54.214051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:54.238397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038fec4-12b5-4fe5-9c74-9aa8b516e667 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:54.242015 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd5430-c263-410e-97a0-6a96404b084b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:54.242611 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd5430-c263-410e-97a0-6a96404b084b service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:54.242888 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd5430-c263-410e-97a0-6a96404b084b service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:54.243151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd5430-c263-410e-97a0-6a96404b084b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:54.252803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd5430-c263-410e-97a0-6a96404b084b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:54.254258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd5430-c263-410e-97a0-6a96404b084b service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:54.263555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1412/2818] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:54.525576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159c833-aaee-4c57-abbf-31a90e73808f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:54.527133 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-b159c833-aaee-4c57-abbf-31a90e73808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:54.542280 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159c833-aaee-4c57-abbf-31a90e73808f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:54.547833 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:54.548371 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:54.548752 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:54.549167 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] PATH_INFO: `/v3/projects/45fddbb738184ecfa8bfb9ea51f61b58` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:54.550166 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45fddbb738184ecfa8bfb9ea51f61b58)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:54.552711 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:54.602111 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45fddbb738184ecfa8bfb9ea51f61b58', 'request_id': 'req-6887d10d-486d-4ad0-80f7-6e21ca905299'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.609219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45fddbb738184ecfa8bfb9ea51f61b58', 'request_id': 'req-6887d10d-486d-4ad0-80f7-6e21ca905299'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.615283 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45fddbb738184ecfa8bfb9ea51f61b58', 'request_id': 'req-6887d10d-486d-4ad0-80f7-6e21ca905299'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.625258 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] The token cache is being invalidate because project 45fddbb738184ecfa8bfb9ea51f61b58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:54.625571 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.626192 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.628171 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1407/2819] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:54 2022] DELETE /identity/v3/projects/45fddbb738184ecfa8bfb9ea51f61b58 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:54.632200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd5430-c263-410e-97a0-6a96404b084b service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:54.641702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd5430-c263-410e-97a0-6a96404b084b service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:54.695322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:54.695963 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:54.696237 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:54.696506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] PATH_INFO: `/v3/users/74acfa4c97d2416e908b9dbcbbd08f06` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:54.697651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74acfa4c97d2416e908b9dbcbbd08f06)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:54.699535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:54.747019 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74acfa4c97d2416e908b9dbcbbd08f06'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.747659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74acfa4c97d2416e908b9dbcbbd08f06'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.754908 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74acfa4c97d2416e908b9dbcbbd08f06'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.766490 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74acfa4c97d2416e908b9dbcbbd08f06'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.774122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74acfa4c97d2416e908b9dbcbbd08f06'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:54.779895 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1413/2820] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:54 2022] DELETE /identity/v3/users/74acfa4c97d2416e908b9dbcbbd08f06 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:55.054456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:55.056410 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:55.074506 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887d10d-486d-4ad0-80f7-6e21ca905299 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:55.137478 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:55.138136 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:55.138327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:55.138595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] PATH_INFO: `/v3/projects/f491af6306ff4747b0d6347c0495c39a` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:55.139700 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f491af6306ff4747b0d6347c0495c39a)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:55.141636 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:55.187823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f491af6306ff4747b0d6347c0495c39a', 'request_id': 'req-24236117-1eb0-4ead-826b-583666d2a239'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.194980 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f491af6306ff4747b0d6347c0495c39a', 'request_id': 'req-24236117-1eb0-4ead-826b-583666d2a239'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.200993 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f491af6306ff4747b0d6347c0495c39a', 'request_id': 'req-24236117-1eb0-4ead-826b-583666d2a239'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.211948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] The token cache is being invalidate because project f491af6306ff4747b0d6347c0495c39a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:55.212232 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.212861 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.214545 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1408/2821] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:55 2022] DELETE /identity/v3/projects/f491af6306ff4747b0d6347c0495c39a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:55.218706 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:55.220174 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:55.233243 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbf388-4542-45a8-bb70-ecbeee564485 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:55.282231 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:55.297707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:55.298749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:55.299055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:55.299371 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] PATH_INFO: `/v3/users/32c4d5ed7a474411bdc9cf0daf35a4eb` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:55.300786 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c4d5ed7a474411bdc9cf0daf35a4eb)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:55.302925 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:55.351217 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c4d5ed7a474411bdc9cf0daf35a4eb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.351871 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c4d5ed7a474411bdc9cf0daf35a4eb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.358698 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c4d5ed7a474411bdc9cf0daf35a4eb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.370290 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c4d5ed7a474411bdc9cf0daf35a4eb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.378065 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-a5a0c352-60e2-43e4-97e3-05463ae618ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c4d5ed7a474411bdc9cf0daf35a4eb'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.383849 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1414/2822] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:55 2022] DELETE /identity/v3/users/32c4d5ed7a474411bdc9cf0daf35a4eb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:55.650853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:55.652397 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:55.667964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24236117-1eb0-4ead-826b-583666d2a239 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:55.672830 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:55.727630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:55.728253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:55.728529 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:55.728799 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] PATH_INFO: `/v3/projects/99df29a35cff47b9b27914dfe5ce9863` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:55.729930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] RBAC: Authorizing `identity:delete_project(project_id=99df29a35cff47b9b27914dfe5ce9863)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:55.731815 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:55.776948 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99df29a35cff47b9b27914dfe5ce9863', 'request_id': 'req-e6fca436-8529-4318-8423-0c18500af22a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.783638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99df29a35cff47b9b27914dfe5ce9863', 'request_id': 'req-e6fca436-8529-4318-8423-0c18500af22a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.789689 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99df29a35cff47b9b27914dfe5ce9863', 'request_id': 'req-e6fca436-8529-4318-8423-0c18500af22a'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.800282 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] The token cache is being invalidate because project 99df29a35cff47b9b27914dfe5ce9863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:55.800617 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.801252 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-e6fca436-8529-4318-8423-0c18500af22a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:55.802731 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1409/2823] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:55 2022] DELETE /identity/v3/projects/99df29a35cff47b9b27914dfe5ce9863 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:55.807661 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ea1f6-698e-48bd-8521-bdf333596077 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:55.807971 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ea1f6-698e-48bd-8521-bdf333596077 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:55.808244 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ea1f6-698e-48bd-8521-bdf333596077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:55.827662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-772ea1f6-698e-48bd-8521-bdf333596077 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:55.828852 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-772ea1f6-698e-48bd-8521-bdf333596077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:55.885177 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1415/2824] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:55.894169 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a13a8-2978-4ce5-b22a-b611f102a181 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:55.894460 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a13a8-2978-4ce5-b22a-b611f102a181 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:55.894740 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a13a8-2978-4ce5-b22a-b611f102a181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:55.914150 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-2d3a13a8-2978-4ce5-b22a-b611f102a181 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:55.915336 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-2d3a13a8-2978-4ce5-b22a-b611f102a181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:55.922995 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1410/2825] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:55.927647 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ea1f6-698e-48bd-8521-bdf333596077 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:55.936972 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ea1f6-698e-48bd-8521-bdf333596077 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:55.942035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16e618-d3da-45bf-9b15-5153982fcea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:55.942624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16e618-d3da-45bf-9b15-5153982fcea6 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:55.942896 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16e618-d3da-45bf-9b15-5153982fcea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:55.943171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16e618-d3da-45bf-9b15-5153982fcea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:55.943810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16e618-d3da-45bf-9b15-5153982fcea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:03:55.944110 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16e618-d3da-45bf-9b15-5153982fcea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:55.945713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16e618-d3da-45bf-9b15-5153982fcea6 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:55.956945 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1416/2826] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:03:55.961588 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a13a8-2978-4ce5-b22a-b611f102a181 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:55.970619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a13a8-2978-4ce5-b22a-b611f102a181 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:55.975221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab1e5b-f172-4325-a564-651df5d8c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:55.975818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab1e5b-f172-4325-a564-651df5d8c225 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:55.976099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab1e5b-f172-4325-a564-651df5d8c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:55.976372 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab1e5b-f172-4325-a564-651df5d8c225 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:55.976999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab1e5b-f172-4325-a564-651df5d8c225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:55.978924 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab1e5b-f172-4325-a564-651df5d8c225 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:56.006677 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1411/2827] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:03:55 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:56.012157 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16e618-d3da-45bf-9b15-5153982fcea6 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:56.022882 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16e618-d3da-45bf-9b15-5153982fcea6 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:56.076552 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094738da-6ba7-43f4-be95-9bdacba88738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:56.077148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094738da-6ba7-43f4-be95-9bdacba88738 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:56.077426 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094738da-6ba7-43f4-be95-9bdacba88738 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:56.077736 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094738da-6ba7-43f4-be95-9bdacba88738 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:56.078354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094738da-6ba7-43f4-be95-9bdacba88738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:56.080743 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094738da-6ba7-43f4-be95-9bdacba88738 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:56.088877 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:56.104414 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1417/2828] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:03:56 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:56.108544 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab1e5b-f172-4325-a564-651df5d8c225 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:56.118865 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab1e5b-f172-4325-a564-651df5d8c225 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:56.123270 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8d6f2-1148-47a6-834e-81553c197d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:56.123893 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8d6f2-1148-47a6-834e-81553c197d9c admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:56.124164 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8d6f2-1148-47a6-834e-81553c197d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:56.124433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8d6f2-1148-47a6-834e-81553c197d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:56.125114 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8d6f2-1148-47a6-834e-81553c197d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:56.127177 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8d6f2-1148-47a6-834e-81553c197d9c admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:56.139720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1412/2829] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:56.141559 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094738da-6ba7-43f4-be95-9bdacba88738 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:56.152613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094738da-6ba7-43f4-be95-9bdacba88738 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:56.157617 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba0c0b-cc60-4fb9-9eab-067d23680d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:56.158232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba0c0b-cc60-4fb9-9eab-067d23680d08 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:56.158515 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba0c0b-cc60-4fb9-9eab-067d23680d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:56.158793 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba0c0b-cc60-4fb9-9eab-067d23680d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:56.159440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba0c0b-cc60-4fb9-9eab-067d23680d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:56.161039 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba0c0b-cc60-4fb9-9eab-067d23680d08 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:56.172044 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1418/2830] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:03:56.176183 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8d6f2-1148-47a6-834e-81553c197d9c admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:56.187637 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8d6f2-1148-47a6-834e-81553c197d9c admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:56.194365 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef488ca4-70b4-4652-a8de-1229f1e58673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:56.195018 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488ca4-70b4-4652-a8de-1229f1e58673 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:56.198387 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488ca4-70b4-4652-a8de-1229f1e58673 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:56.198710 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488ca4-70b4-4652-a8de-1229f1e58673 admin admin] PATH_INFO: `/v3/projects/41772d612abc4efa9a7c50a78da63dce/users/44b3d40ccf454abeb3abda102fe97925/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:56.216118 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef488ca4-70b4-4652-a8de-1229f1e58673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41772d612abc4efa9a7c50a78da63dce, user_id=44b3d40ccf454abeb3abda102fe97925, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:56.218805 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef488ca4-70b4-4652-a8de-1229f1e58673 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:56.233540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1413/2831] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:56 2022] PUT /identity/v3/projects/41772d612abc4efa9a7c50a78da63dce/users/44b3d40ccf454abeb3abda102fe97925/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:56.238867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8dab1-4b51-4ada-b2a5-5b3294287ab8 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:56.239316 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8dab1-4b51-4ada-b2a5-5b3294287ab8 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:56.239421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8dab1-4b51-4ada-b2a5-5b3294287ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:56.258613 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-88d8dab1-4b51-4ada-b2a5-5b3294287ab8 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:56.259819 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-88d8dab1-4b51-4ada-b2a5-5b3294287ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:56.318767 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1419/2832] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:56.328466 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef488ca4-70b4-4652-a8de-1229f1e58673 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:56.330812 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-ef488ca4-70b4-4652-a8de-1229f1e58673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:56.369906 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:56.388761 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef488ca4-70b4-4652-a8de-1229f1e58673 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:56.392964 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b165f-d5ae-4414-9a17-42c6c8592f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:56.393594 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b165f-d5ae-4414-9a17-42c6c8592f2a service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:56.393876 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b165f-d5ae-4414-9a17-42c6c8592f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:56.394145 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b165f-d5ae-4414-9a17-42c6c8592f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:56.403020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b165f-d5ae-4414-9a17-42c6c8592f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:56.404445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b165f-d5ae-4414-9a17-42c6c8592f2a service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:56.413818 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1414/2833] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:57.084378 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b083f-477b-4861-9921-8e70f942c818 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:57.084714 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b083f-477b-4861-9921-8e70f942c818 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:57.084934 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b083f-477b-4861-9921-8e70f942c818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:57.115721 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-3f3b083f-477b-4861-9921-8e70f942c818 None None] MFA Rules not processed for user `44b3d40ccf454abeb3abda102fe97925`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:57.116975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3f3b083f-477b-4861-9921-8e70f942c818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:57.185923 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1420/2834] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:57.196839 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b165f-d5ae-4414-9a17-42c6c8592f2a service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:57.198570 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-849b165f-d5ae-4414-9a17-42c6c8592f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:57.257374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b165f-d5ae-4414-9a17-42c6c8592f2a service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:57.260632 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c82419-2069-4d8e-b3be-4c56ea95fd47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:57.261232 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c82419-2069-4d8e-b3be-4c56ea95fd47 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:57.261482 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c82419-2069-4d8e-b3be-4c56ea95fd47 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:57.261854 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c82419-2069-4d8e-b3be-4c56ea95fd47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:57.271099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c82419-2069-4d8e-b3be-4c56ea95fd47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:57.272658 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c82419-2069-4d8e-b3be-4c56ea95fd47 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:57.298680 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1415/2835] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:58.421256 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b083f-477b-4861-9921-8e70f942c818 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:58.422690 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3f3b083f-477b-4861-9921-8e70f942c818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:58.436621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b083f-477b-4861-9921-8e70f942c818 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:58.441061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:58.441711 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:58.441993 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:58.442265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] PATH_INFO: `/v3/users/ab8bb337acf046b5a3038131185d7427` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:58.443383 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab8bb337acf046b5a3038131185d7427)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:58.445295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:58.491964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab8bb337acf046b5a3038131185d7427'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:58.492601 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab8bb337acf046b5a3038131185d7427'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:58.499506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab8bb337acf046b5a3038131185d7427'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:58.505077 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:58.513777 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab8bb337acf046b5a3038131185d7427'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:58.520725 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab8bb337acf046b5a3038131185d7427'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:58.526626 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1421/2836] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:03:58 2022] DELETE /identity/v3/users/ab8bb337acf046b5a3038131185d7427 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:58.531124 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa16cd5-aae4-42fa-9721-5694bf2f3b80 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:58.531583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa16cd5-aae4-42fa-9721-5694bf2f3b80 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:58.531681 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa16cd5-aae4-42fa-9721-5694bf2f3b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:58.551344 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-7aa16cd5-aae4-42fa-9721-5694bf2f3b80 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:58.552539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7aa16cd5-aae4-42fa-9721-5694bf2f3b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:58.609802 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1416/2837] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:03:58.620206 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:58.621668 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:58.672651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702344f4-f497-4b09-80e0-59f7c9d3747c None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:58.672947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702344f4-f497-4b09-80e0-59f7c9d3747c None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:58.673231 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702344f4-f497-4b09-80e0-59f7c9d3747c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:58.679624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2a7ab-b080-4849-922e-df234f050752 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:58.684509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52537c2-3906-45a6-b0e1-e854df8288d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:58.685111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52537c2-3906-45a6-b0e1-e854df8288d5 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:58.685420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52537c2-3906-45a6-b0e1-e854df8288d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:58.685728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52537c2-3906-45a6-b0e1-e854df8288d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:58.693819 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-702344f4-f497-4b09-80e0-59f7c9d3747c None None] MFA Rules not processed for user `44b3d40ccf454abeb3abda102fe97925`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:58.695028 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-702344f4-f497-4b09-80e0-59f7c9d3747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:58.696082 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52537c2-3906-45a6-b0e1-e854df8288d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:58.698377 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52537c2-3906-45a6-b0e1-e854df8288d5 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:58.712658 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1422/2838] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:03:58.765289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1417/2839] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:03:58.995375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52537c2-3906-45a6-b0e1-e854df8288d5 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:58.997832 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a52537c2-3906-45a6-b0e1-e854df8288d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:59.013733 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702344f4-f497-4b09-80e0-59f7c9d3747c None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:59.014326 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52537c2-3906-45a6-b0e1-e854df8288d5 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:59.015526 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-702344f4-f497-4b09-80e0-59f7c9d3747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:59.019899 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:59.020581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.020868 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.021147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] PATH_INFO: `/v3/projects/47f39025c72e44f4a49e0aed7ef26073` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.022465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f39025c72e44f4a49e0aed7ef26073)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:59.024373 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:59.030675 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702344f4-f497-4b09-80e0-59f7c9d3747c None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:59.035118 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce30ba6-d997-48ac-8b74-e44e574ae2e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:59.035723 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce30ba6-d997-48ac-8b74-e44e574ae2e7 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.036001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce30ba6-d997-48ac-8b74-e44e574ae2e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.036271 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce30ba6-d997-48ac-8b74-e44e574ae2e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.037985 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-bce30ba6-d997-48ac-8b74-e44e574ae2e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:59.047684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce30ba6-d997-48ac-8b74-e44e574ae2e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:59.049157 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce30ba6-d997-48ac-8b74-e44e574ae2e7 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:59.060099 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1418/2840] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:03:59.073121 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f39025c72e44f4a49e0aed7ef26073', 'request_id': 'req-38889b70-cd41-4b3a-9648-e4a988d0142d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:59.081550 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f39025c72e44f4a49e0aed7ef26073', 'request_id': 'req-38889b70-cd41-4b3a-9648-e4a988d0142d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:59.088429 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f39025c72e44f4a49e0aed7ef26073', 'request_id': 'req-38889b70-cd41-4b3a-9648-e4a988d0142d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:59.100147 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] The token cache is being invalidate because project 47f39025c72e44f4a49e0aed7ef26073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:03:59.100504 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:59.101264 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-38889b70-cd41-4b3a-9648-e4a988d0142d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:03:59.102783 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1423/2841] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:03:58 2022] DELETE /identity/v3/projects/47f39025c72e44f4a49e0aed7ef26073 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:59.107644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7f925-f339-4f2d-8e1b-22bc2640e4eb None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.107930 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7f925-f339-4f2d-8e1b-22bc2640e4eb None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.108201 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7f925-f339-4f2d-8e1b-22bc2640e4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.128503 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-80b7f925-f339-4f2d-8e1b-22bc2640e4eb None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:59.129748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-80b7f925-f339-4f2d-8e1b-22bc2640e4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:59.138176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=100 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:59.191745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1419/2842] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:03:59.199489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af456ab-7dc2-47b1-b18f-8f2e396af8ff None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.199787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af456ab-7dc2-47b1-b18f-8f2e396af8ff None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.200064 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af456ab-7dc2-47b1-b18f-8f2e396af8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.220399 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-1af456ab-7dc2-47b1-b18f-8f2e396af8ff None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:59.221674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-1af456ab-7dc2-47b1-b18f-8f2e396af8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:59.229514 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1424/2843] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:59.233795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7f925-f339-4f2d-8e1b-22bc2640e4eb None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:59.243394 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7f925-f339-4f2d-8e1b-22bc2640e4eb None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:59.248049 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7969e44-b215-4c25-bba1-2d8fce4de678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:59.248641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7969e44-b215-4c25-bba1-2d8fce4de678 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.248916 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7969e44-b215-4c25-bba1-2d8fce4de678 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.249182 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7969e44-b215-4c25-bba1-2d8fce4de678 admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.249856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7969e44-b215-4c25-bba1-2d8fce4de678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:03:59.250152 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7969e44-b215-4c25-bba1-2d8fce4de678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:59.252261 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7969e44-b215-4c25-bba1-2d8fce4de678 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:59.264264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1420/2844] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:03:59.269133 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af456ab-7dc2-47b1-b18f-8f2e396af8ff None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:59.278537 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af456ab-7dc2-47b1-b18f-8f2e396af8ff None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:59.284216 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197b64a-6d33-4c84-ba41-48b0f993cdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:59.284878 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197b64a-6d33-4c84-ba41-48b0f993cdf0 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.285219 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197b64a-6d33-4c84-ba41-48b0f993cdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.285434 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197b64a-6d33-4c84-ba41-48b0f993cdf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.286234 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197b64a-6d33-4c84-ba41-48b0f993cdf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:59.288294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197b64a-6d33-4c84-ba41-48b0f993cdf0 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:59.321453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1425/2845] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:03:59 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:59.322807 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7969e44-b215-4c25-bba1-2d8fce4de678 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:59.333969 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7969e44-b215-4c25-bba1-2d8fce4de678 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:59.405955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774b266-48ca-4a7c-83e8-60b8f4887857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:59.406564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774b266-48ca-4a7c-83e8-60b8f4887857 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.406846 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774b266-48ca-4a7c-83e8-60b8f4887857 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.407120 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774b266-48ca-4a7c-83e8-60b8f4887857 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.407754 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774b266-48ca-4a7c-83e8-60b8f4887857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:59.409743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774b266-48ca-4a7c-83e8-60b8f4887857 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:59.430428 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1421/2846] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:03:59 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:03:59.434312 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197b64a-6d33-4c84-ba41-48b0f993cdf0 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:59.444936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197b64a-6d33-4c84-ba41-48b0f993cdf0 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:59.449456 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d85c5-41b5-44b4-8261-8fa39777a8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:59.450081 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d85c5-41b5-44b4-8261-8fa39777a8ee admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.450361 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d85c5-41b5-44b4-8261-8fa39777a8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.450633 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d85c5-41b5-44b4-8261-8fa39777a8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.451270 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d85c5-41b5-44b4-8261-8fa39777a8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:59.452853 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d85c5-41b5-44b4-8261-8fa39777a8ee admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:59.463985 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1426/2847] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:59.467838 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774b266-48ca-4a7c-83e8-60b8f4887857 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:59.477951 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774b266-48ca-4a7c-83e8-60b8f4887857 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:59.482621 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1f086-44fc-4017-aca8-b2a40e877d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:59.483211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1f086-44fc-4017-aca8-b2a40e877d44 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.483543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1f086-44fc-4017-aca8-b2a40e877d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.483795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1f086-44fc-4017-aca8-b2a40e877d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.484437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1f086-44fc-4017-aca8-b2a40e877d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:59.486568 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1f086-44fc-4017-aca8-b2a40e877d44 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:59.496947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1422/2848] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:03:59.500751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d85c5-41b5-44b4-8261-8fa39777a8ee admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:59.511136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d85c5-41b5-44b4-8261-8fa39777a8ee admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:59.515701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9b026-fa40-4621-9202-c45149fc2bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:59.516365 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9b026-fa40-4621-9202-c45149fc2bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.516612 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9b026-fa40-4621-9202-c45149fc2bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.516918 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9b026-fa40-4621-9202-c45149fc2bd8 admin admin] PATH_INFO: `/v3/projects/099f0cebfae0459d9bac06c467203d83/users/bcbccfd9466944e6b356583c98568849/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.531421 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9b026-fa40-4621-9202-c45149fc2bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=099f0cebfae0459d9bac06c467203d83, user_id=bcbccfd9466944e6b356583c98568849, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:59.534176 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9b026-fa40-4621-9202-c45149fc2bd8 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:59.546730 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1427/2849] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:03:59 2022] PUT /identity/v3/projects/099f0cebfae0459d9bac06c467203d83/users/bcbccfd9466944e6b356583c98568849/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:03:59.612481 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09154e08-019f-4693-a2db-0aae8cecf329 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.613083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09154e08-019f-4693-a2db-0aae8cecf329 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.613083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09154e08-019f-4693-a2db-0aae8cecf329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.641047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-09154e08-019f-4693-a2db-0aae8cecf329 None None] MFA Rules not processed for user `bcbccfd9466944e6b356583c98568849`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:03:59.642312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-09154e08-019f-4693-a2db-0aae8cecf329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:59.703949 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1423/2850] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:03:59.715261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9b026-fa40-4621-9202-c45149fc2bd8 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:03:59.716933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-2ba9b026-fa40-4621-9202-c45149fc2bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:03:59.722247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:03:59.775100 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9b026-fa40-4621-9202-c45149fc2bd8 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:03:59.778751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bab23-bee9-47af-a46a-91947e9c8d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:03:59.779358 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bab23-bee9-47af-a46a-91947e9c8d83 service cinder] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:03:59.779641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bab23-bee9-47af-a46a-91947e9c8d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:03:59.779920 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bab23-bee9-47af-a46a-91947e9c8d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:03:59.788035 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bab23-bee9-47af-a46a-91947e9c8d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:03:59.789466 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bab23-bee9-47af-a46a-91947e9c8d83 service cinder] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:03:59.798148 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1428/2851] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:03:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:04:00.416744 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09154e08-019f-4693-a2db-0aae8cecf329 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:00.418374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-09154e08-019f-4693-a2db-0aae8cecf329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:00.429154 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:00.496081 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09154e08-019f-4693-a2db-0aae8cecf329 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:00.501084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12470d0f-49d0-4ad0-987a-798677ef713c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:00.501724 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12470d0f-49d0-4ad0-987a-798677ef713c service placement] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:00.502005 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12470d0f-49d0-4ad0-987a-798677ef713c service placement] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:00.502279 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12470d0f-49d0-4ad0-987a-798677ef713c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:00.504050 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-12470d0f-49d0-4ad0-987a-798677ef713c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:00.515295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12470d0f-49d0-4ad0-987a-798677ef713c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:00.516871 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12470d0f-49d0-4ad0-987a-798677ef713c service placement] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:00.527186 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1424/2852] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:04:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:04:02.360742 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bab23-bee9-47af-a46a-91947e9c8d83 service cinder] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:02.425009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bab23-bee9-47af-a46a-91947e9c8d83 service cinder] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:02.430795 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:02.431432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:02.431712 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:02.431987 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] PATH_INFO: `/v3/users/bcbccfd9466944e6b356583c98568849` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:02.433340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcbccfd9466944e6b356583c98568849)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:02.435195 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:02.482295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcbccfd9466944e6b356583c98568849'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:02.482935 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcbccfd9466944e6b356583c98568849'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:02.490555 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcbccfd9466944e6b356583c98568849'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:02.503172 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcbccfd9466944e6b356583c98568849'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:02.512240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcbccfd9466944e6b356583c98568849'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:02.518505 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1429/2853] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:04:02 2022] DELETE /identity/v3/users/bcbccfd9466944e6b356583c98568849 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:04:02.523662 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e2467-9d8f-4c46-b392-1dfc7dd62f77 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:02.524149 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e2467-9d8f-4c46-b392-1dfc7dd62f77 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:02.524251 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e2467-9d8f-4c46-b392-1dfc7dd62f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:02.542929 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-980e2467-9d8f-4c46-b392-1dfc7dd62f77 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:02.544136 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-980e2467-9d8f-4c46-b392-1dfc7dd62f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:02.602003 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1425/2854] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:02.616939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:02.618603 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:02.653191 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:02.685897 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a3362-00b9-469d-8b0c-47c298657127 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:02.690120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee548aa-3edc-4eef-963f-d97f219d5da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:02.690741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee548aa-3edc-4eef-963f-d97f219d5da0 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:02.691022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee548aa-3edc-4eef-963f-d97f219d5da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:02.691294 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee548aa-3edc-4eef-963f-d97f219d5da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:02.700651 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee548aa-3edc-4eef-963f-d97f219d5da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:02.702140 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee548aa-3edc-4eef-963f-d97f219d5da0 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:02.712430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1430/2855] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:04:03.120709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e2467-9d8f-4c46-b392-1dfc7dd62f77 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:03.122684 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-980e2467-9d8f-4c46-b392-1dfc7dd62f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:03.136488 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e2467-9d8f-4c46-b392-1dfc7dd62f77 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:03.141748 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:03.142437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.142717 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.143000 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] PATH_INFO: `/v3/projects/099f0cebfae0459d9bac06c467203d83` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.144199 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] RBAC: Authorizing `identity:delete_project(project_id=099f0cebfae0459d9bac06c467203d83)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:03.146077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:03.200083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '099f0cebfae0459d9bac06c467203d83', 'request_id': 'req-33485cc4-f587-467f-8aae-b336d7a1061d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:03.208374 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '099f0cebfae0459d9bac06c467203d83', 'request_id': 'req-33485cc4-f587-467f-8aae-b336d7a1061d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:03.215456 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '099f0cebfae0459d9bac06c467203d83', 'request_id': 'req-33485cc4-f587-467f-8aae-b336d7a1061d'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:03.227030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] The token cache is being invalidate because project 099f0cebfae0459d9bac06c467203d83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:04:03.227315 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:03.227955 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-33485cc4-f587-467f-8aae-b336d7a1061d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:03.229454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1426/2856] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:04:03 2022] DELETE /identity/v3/projects/099f0cebfae0459d9bac06c467203d83 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:04:03.234276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e22adc-1e86-41ac-8c93-953279e36043 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.234569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e22adc-1e86-41ac-8c93-953279e36043 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.234856 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e22adc-1e86-41ac-8c93-953279e36043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.254939 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-23e22adc-1e86-41ac-8c93-953279e36043 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:03.256171 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-23e22adc-1e86-41ac-8c93-953279e36043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:03.322433 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1431/2857] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:03.330602 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff5b0f-1055-48fe-b4e9-fd19f0e23492 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.330922 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff5b0f-1055-48fe-b4e9-fd19f0e23492 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.331196 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff5b0f-1055-48fe-b4e9-fd19f0e23492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.353517 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-62ff5b0f-1055-48fe-b4e9-fd19f0e23492 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:03.354801 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-62ff5b0f-1055-48fe-b4e9-fd19f0e23492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:03.363178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1427/2858] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:03.367492 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e22adc-1e86-41ac-8c93-953279e36043 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:03.377198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e22adc-1e86-41ac-8c93-953279e36043 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:03.381749 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e67061-263d-4142-95c9-4e3e0fe885ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:03.382340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67061-263d-4142-95c9-4e3e0fe885ea admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.382620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67061-263d-4142-95c9-4e3e0fe885ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.382893 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67061-263d-4142-95c9-4e3e0fe885ea admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.383539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e67061-263d-4142-95c9-4e3e0fe885ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:04:03.383964 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e67061-263d-4142-95c9-4e3e0fe885ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:03.385579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e67061-263d-4142-95c9-4e3e0fe885ea admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:03.397450 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1432/2859] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:04:03.401583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff5b0f-1055-48fe-b4e9-fd19f0e23492 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:03.411314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff5b0f-1055-48fe-b4e9-fd19f0e23492 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:03.415787 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b63608-a543-4260-b18e-ce7b199886f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:03.416368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b63608-a543-4260-b18e-ce7b199886f1 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.416639 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b63608-a543-4260-b18e-ce7b199886f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.416902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b63608-a543-4260-b18e-ce7b199886f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.417557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b63608-a543-4260-b18e-ce7b199886f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:03.419441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b63608-a543-4260-b18e-ce7b199886f1 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:03.433395 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:03.447130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1428/2860] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:04:03 2022] POST /identity/v3/projects => generated 395 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:04:03.451285 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e67061-263d-4142-95c9-4e3e0fe885ea admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:03.462281 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e67061-263d-4142-95c9-4e3e0fe885ea admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:03.520942 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e669914-501e-499f-84a8-9557cc1e1fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:03.521674 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e669914-501e-499f-84a8-9557cc1e1fbd admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.521965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e669914-501e-499f-84a8-9557cc1e1fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.522247 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e669914-501e-499f-84a8-9557cc1e1fbd admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.522891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e669914-501e-499f-84a8-9557cc1e1fbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:03.524810 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e669914-501e-499f-84a8-9557cc1e1fbd admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:03.551579 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1433/2861] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:04:03 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:04:03.552792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b63608-a543-4260-b18e-ce7b199886f1 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:03.563040 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b63608-a543-4260-b18e-ce7b199886f1 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:03.567667 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd7fe3-ec90-48b5-841b-641ec9f1777a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:03.568264 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd7fe3-ec90-48b5-841b-641ec9f1777a admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.568540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd7fe3-ec90-48b5-841b-641ec9f1777a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.568811 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd7fe3-ec90-48b5-841b-641ec9f1777a admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.569462 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd7fe3-ec90-48b5-841b-641ec9f1777a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:03.571088 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd7fe3-ec90-48b5-841b-641ec9f1777a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:03.583441 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1429/2862] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:04:03.587826 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e669914-501e-499f-84a8-9557cc1e1fbd admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:03.599833 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e669914-501e-499f-84a8-9557cc1e1fbd admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:03.605526 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50b499-9a06-48d4-9a61-e8c65d80b3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:03.605769 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50b499-9a06-48d4-9a61-e8c65d80b3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.606057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50b499-9a06-48d4-9a61-e8c65d80b3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.606335 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50b499-9a06-48d4-9a61-e8c65d80b3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.606989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50b499-9a06-48d4-9a61-e8c65d80b3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:03.608587 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50b499-9a06-48d4-9a61-e8c65d80b3c7 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:03.624779 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1434/2863] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:04:03.625896 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd7fe3-ec90-48b5-841b-641ec9f1777a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:03.636943 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd7fe3-ec90-48b5-841b-641ec9f1777a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:03.642052 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bf6b8-7daa-471f-9e23-abf9683301e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:03.643297 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf6b8-7daa-471f-9e23-abf9683301e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.643575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf6b8-7daa-471f-9e23-abf9683301e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.643849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf6b8-7daa-471f-9e23-abf9683301e0 admin admin] PATH_INFO: `/v3/projects/bac22c6681cd4d6a8778fc0534e3252b/users/df3d38aa54a04f7aa365e5c1ba359e02/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.660419 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bf6b8-7daa-471f-9e23-abf9683301e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bac22c6681cd4d6a8778fc0534e3252b, user_id=df3d38aa54a04f7aa365e5c1ba359e02, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:03.663178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bf6b8-7daa-471f-9e23-abf9683301e0 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:03.677349 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1430/2864] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:04:03 2022] PUT /identity/v3/projects/bac22c6681cd4d6a8778fc0534e3252b/users/df3d38aa54a04f7aa365e5c1ba359e02/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:04:03.687480 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed782a28-305f-436e-8559-63298c6b23c2 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.687765 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed782a28-305f-436e-8559-63298c6b23c2 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.688070 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed782a28-305f-436e-8559-63298c6b23c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.716728 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-ed782a28-305f-436e-8559-63298c6b23c2 None None] MFA Rules not processed for user `df3d38aa54a04f7aa365e5c1ba359e02`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:03.718018 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ed782a28-305f-436e-8559-63298c6b23c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:03.738509 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:03.782139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1435/2865] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:03.794074 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bf6b8-7daa-471f-9e23-abf9683301e0 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:03.795720 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-113bf6b8-7daa-471f-9e23-abf9683301e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:03.851010 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bf6b8-7daa-471f-9e23-abf9683301e0 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:03.854590 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862f44c-f258-47ba-bd61-7b65bf8feaa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:03.855176 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862f44c-f258-47ba-bd61-7b65bf8feaa8 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:03.855450 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862f44c-f258-47ba-bd61-7b65bf8feaa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:03.855715 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862f44c-f258-47ba-bd61-7b65bf8feaa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:03.865578 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862f44c-f258-47ba-bd61-7b65bf8feaa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:03.867019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862f44c-f258-47ba-bd61-7b65bf8feaa8 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:03.877749 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1431/2866] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:04:03 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:04:12.285261 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed782a28-305f-436e-8559-63298c6b23c2 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:12.286933 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-ed782a28-305f-436e-8559-63298c6b23c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:12.367189 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed782a28-305f-436e-8559-63298c6b23c2 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:12.372581 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32af62c-c1a6-446d-b954-f4bb72ae863a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:12.373185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32af62c-c1a6-446d-b954-f4bb72ae863a service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:12.373462 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32af62c-c1a6-446d-b954-f4bb72ae863a service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:12.373759 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32af62c-c1a6-446d-b954-f4bb72ae863a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:12.375464 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-a32af62c-c1a6-446d-b954-f4bb72ae863a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:12.387704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32af62c-c1a6-446d-b954-f4bb72ae863a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:12.389265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32af62c-c1a6-446d-b954-f4bb72ae863a service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:12.399560 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1436/2867] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:04:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:04:12.976365 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862f44c-f258-47ba-bd61-7b65bf8feaa8 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:13.035880 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862f44c-f258-47ba-bd61-7b65bf8feaa8 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:13.040314 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:13.040917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:13.041190 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:13.041454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] PATH_INFO: `/v3/users/df3d38aa54a04f7aa365e5c1ba359e02` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:13.042580 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df3d38aa54a04f7aa365e5c1ba359e02)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:13.044479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:13.088444 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df3d38aa54a04f7aa365e5c1ba359e02'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:13.089093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df3d38aa54a04f7aa365e5c1ba359e02'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:13.095974 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df3d38aa54a04f7aa365e5c1ba359e02'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:13.103714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df3d38aa54a04f7aa365e5c1ba359e02'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:13.104561 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df3d38aa54a04f7aa365e5c1ba359e02'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:13.105795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:13.124604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1432/2868] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:04:12 2022] DELETE /identity/v3/users/df3d38aa54a04f7aa365e5c1ba359e02 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:04:13.129153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d5471-0f1c-4fd2-a604-49f46889c6b4 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:13.129440 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d5471-0f1c-4fd2-a604-49f46889c6b4 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:13.129748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d5471-0f1c-4fd2-a604-49f46889c6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:13.148444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-d72d5471-0f1c-4fd2-a604-49f46889c6b4 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:13.149707 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d72d5471-0f1c-4fd2-a604-49f46889c6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:13.207324 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1437/2869] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:13.217129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:13.218644 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:13.279333 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4878e-640f-4ad9-8e2a-f3c67d44f356 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:13.286575 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea04f5a-dd3f-4114-bd77-164bc610c6a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:13.287194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea04f5a-dd3f-4114-bd77-164bc610c6a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:13.287474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea04f5a-dd3f-4114-bd77-164bc610c6a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:13.287747 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea04f5a-dd3f-4114-bd77-164bc610c6a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:13.298089 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea04f5a-dd3f-4114-bd77-164bc610c6a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:13.299543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea04f5a-dd3f-4114-bd77-164bc610c6a2 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:13.309304 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1433/2870] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:04:13.581869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d5471-0f1c-4fd2-a604-49f46889c6b4 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:13.583636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d72d5471-0f1c-4fd2-a604-49f46889c6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:13.596122 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d5471-0f1c-4fd2-a604-49f46889c6b4 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:13.601339 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:13.602146 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:13.602427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:13.602859 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] PATH_INFO: `/v3/projects/bac22c6681cd4d6a8778fc0534e3252b` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:13.604141 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bac22c6681cd4d6a8778fc0534e3252b)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:13.606444 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:13.651029 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:13.655522 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bac22c6681cd4d6a8778fc0534e3252b', 'request_id': 'req-509bab26-74ee-4ae5-8eb9-481e40a96aa1'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:13.663273 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bac22c6681cd4d6a8778fc0534e3252b', 'request_id': 'req-509bab26-74ee-4ae5-8eb9-481e40a96aa1'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:13.669622 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bac22c6681cd4d6a8778fc0534e3252b', 'request_id': 'req-509bab26-74ee-4ae5-8eb9-481e40a96aa1'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:13.679788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] The token cache is being invalidate because project bac22c6681cd4d6a8778fc0534e3252b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:04:13.680072 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:13.680694 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-509bab26-74ee-4ae5-8eb9-481e40a96aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:13.682185 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1438/2871] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:04:13 2022] DELETE /identity/v3/projects/bac22c6681cd4d6a8778fc0534e3252b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:04:18.715907 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97faa7-2eef-424b-83cf-c0f1d3f70689 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:18.716382 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97faa7-2eef-424b-83cf-c0f1d3f70689 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:18.716486 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97faa7-2eef-424b-83cf-c0f1d3f70689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:18.735601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-9b97faa7-2eef-424b-83cf-c0f1d3f70689 None None] MFA Rules not processed for user `44b3d40ccf454abeb3abda102fe97925`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:18.736792 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9b97faa7-2eef-424b-83cf-c0f1d3f70689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:18.789753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1434/2872] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:23.676803 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=8,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:28.782875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=58,keystone:DELETE=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:41.182269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48336db1-a6f6-4d1d-b5e8-0ae37cf515e4 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:41.182763 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48336db1-a6f6-4d1d-b5e8-0ae37cf515e4 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:41.182869 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48336db1-a6f6-4d1d-b5e8-0ae37cf515e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:41.231311 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-48336db1-a6f6-4d1d-b5e8-0ae37cf515e4 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:41.232524 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-48336db1-a6f6-4d1d-b5e8-0ae37cf515e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:41.296164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1439/2873] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:41.342388 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97faa7-2eef-424b-83cf-c0f1d3f70689 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:41.343769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-9b97faa7-2eef-424b-83cf-c0f1d3f70689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:41.433687 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97faa7-2eef-424b-83cf-c0f1d3f70689 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:41.438579 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4469110-d3d1-4879-a3b9-85a4802536f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:41.439180 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4469110-d3d1-4879-a3b9-85a4802536f8 service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:41.439454 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4469110-d3d1-4879-a3b9-85a4802536f8 service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:41.439721 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4469110-d3d1-4879-a3b9-85a4802536f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:41.447574 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4469110-d3d1-4879-a3b9-85a4802536f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:41.449127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4469110-d3d1-4879-a3b9-85a4802536f8 service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:41.457369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1435/2874] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 15:04:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:04:51.293420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=40 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:51.453873 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=41 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:56.806733 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab18282-b246-4e07-a760-f8df2fb69ceb None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:56.807200 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab18282-b246-4e07-a760-f8df2fb69ceb None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:56.807295 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab18282-b246-4e07-a760-f8df2fb69ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:56.827401 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6ab18282-b246-4e07-a760-f8df2fb69ceb None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:56.828590 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6ab18282-b246-4e07-a760-f8df2fb69ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:56.836604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1440/2875] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 3 15:04:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:56.848435 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4469110-d3d1-4879-a3b9-85a4802536f8 service swift] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:56.858745 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4469110-d3d1-4879-a3b9-85a4802536f8 service swift] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:56.863750 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ee01b-decf-4898-8f3f-e33c2b63b946 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:56.864327 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ee01b-decf-4898-8f3f-e33c2b63b946 service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:56.864601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ee01b-decf-4898-8f3f-e33c2b63b946 service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:56.864866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ee01b-decf-4898-8f3f-e33c2b63b946 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:56.872557 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ee01b-decf-4898-8f3f-e33c2b63b946 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:56.874142 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ee01b-decf-4898-8f3f-e33c2b63b946 service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:56.882363 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1436/2876] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 15:04:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:04:58.769368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab18282-b246-4e07-a760-f8df2fb69ceb None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:58.770886 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6ab18282-b246-4e07-a760-f8df2fb69ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:58.835578 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab18282-b246-4e07-a760-f8df2fb69ceb None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:58.840030 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:58.840655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:58.840938 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:58.841208 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] PATH_INFO: `/v3/users/44b3d40ccf454abeb3abda102fe97925` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:58.842356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b3d40ccf454abeb3abda102fe97925)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:58.844265 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:58.890835 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b3d40ccf454abeb3abda102fe97925'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:58.891413 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b3d40ccf454abeb3abda102fe97925'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:58.898115 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b3d40ccf454abeb3abda102fe97925'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:58.909681 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b3d40ccf454abeb3abda102fe97925'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:58.916636 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-36b368a4-89d1-48b5-ae23-7b6dfb281ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b3d40ccf454abeb3abda102fe97925'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:58.922432 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1441/2877] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:04:58 2022] DELETE /identity/v3/users/44b3d40ccf454abeb3abda102fe97925 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:04:59.088822 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ee01b-decf-4898-8f3f-e33c2b63b946 service swift] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:59.104330 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ee01b-decf-4898-8f3f-e33c2b63b946 service swift] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:59.161280 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:59.162219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.162219 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.162479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] PATH_INFO: `/v3/projects/41772d612abc4efa9a7c50a78da63dce` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.163604 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41772d612abc4efa9a7c50a78da63dce)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:59.165522 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:59.213220 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41772d612abc4efa9a7c50a78da63dce', 'request_id': 'req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:59.220656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41772d612abc4efa9a7c50a78da63dce', 'request_id': 'req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:59.226786 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41772d612abc4efa9a7c50a78da63dce', 'request_id': 'req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:59.236769 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] The token cache is being invalidate because project 41772d612abc4efa9a7c50a78da63dce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87831) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:04:59.237048 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:59.237696 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-5b0a555f-9e7e-43ac-9bfa-9459a50d3825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:04:59.239149 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1437/2878] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:04:59 2022] DELETE /identity/v3/projects/41772d612abc4efa9a7c50a78da63dce => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:04:59.243949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d763b-67e4-435c-b2ba-56a1f159a4b5 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.244232 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d763b-67e4-435c-b2ba-56a1f159a4b5 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.244502 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d763b-67e4-435c-b2ba-56a1f159a4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.264539 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-178d763b-67e4-435c-b2ba-56a1f159a4b5 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:59.265807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-178d763b-67e4-435c-b2ba-56a1f159a4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:59.323986 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1442/2879] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:59.332221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42e68-ca1c-4a44-bea4-75e14035710e None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.332510 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42e68-ca1c-4a44-bea4-75e14035710e None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.332781 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42e68-ca1c-4a44-bea4-75e14035710e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.352499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-40a42e68-ca1c-4a44-bea4-75e14035710e None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:59.353717 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-40a42e68-ca1c-4a44-bea4-75e14035710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:59.361512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1438/2880] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:59.365821 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d763b-67e4-435c-b2ba-56a1f159a4b5 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:59.375445 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d763b-67e4-435c-b2ba-56a1f159a4b5 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:59.380528 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7701051-1dcb-434c-8417-dc7f0c942c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:59.381155 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7701051-1dcb-434c-8417-dc7f0c942c1c admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.381438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7701051-1dcb-434c-8417-dc7f0c942c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.381745 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7701051-1dcb-434c-8417-dc7f0c942c1c admin admin] PATH_INFO: `/v3/domains` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.382427 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7701051-1dcb-434c-8417-dc7f0c942c1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87830) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:04:59.382732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7701051-1dcb-434c-8417-dc7f0c942c1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:59.384322 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7701051-1dcb-434c-8417-dc7f0c942c1c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:59.392825 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:59.397645 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1443/2881] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:04:59.401080 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a42e68-ca1c-4a44-bea4-75e14035710e None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:59.410654 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a42e68-ca1c-4a44-bea4-75e14035710e None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:59.415877 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945000fa-ee9b-4474-bea4-a4d29bd6ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:59.416420 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945000fa-ee9b-4474-bea4-a4d29bd6ad98 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.416701 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945000fa-ee9b-4474-bea4-a4d29bd6ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.416971 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945000fa-ee9b-4474-bea4-a4d29bd6ad98 admin admin] PATH_INFO: `/v3/projects` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.417651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945000fa-ee9b-4474-bea4-a4d29bd6ad98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:59.419546 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945000fa-ee9b-4474-bea4-a4d29bd6ad98 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:59.446446 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1439/2882] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:04:59 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:04:59.450618 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7701051-1dcb-434c-8417-dc7f0c942c1c admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:59.461292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7701051-1dcb-434c-8417-dc7f0c942c1c admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:59.517051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbd80c-f77d-4456-8e93-0a5cf10f64c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:59.517686 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbd80c-f77d-4456-8e93-0a5cf10f64c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.517965 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbd80c-f77d-4456-8e93-0a5cf10f64c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.518238 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbd80c-f77d-4456-8e93-0a5cf10f64c2 admin admin] PATH_INFO: `/v3/users` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.519355 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbd80c-f77d-4456-8e93-0a5cf10f64c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:59.521375 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbd80c-f77d-4456-8e93-0a5cf10f64c2 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:59.542497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1444/2883] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:04:59 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:04:59.546536 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945000fa-ee9b-4474-bea4-a4d29bd6ad98 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:59.556743 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945000fa-ee9b-4474-bea4-a4d29bd6ad98 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:59.561222 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be279e-e85b-47e5-af64-48042cdc5b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:59.561849 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be279e-e85b-47e5-af64-48042cdc5b81 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.562127 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be279e-e85b-47e5-af64-48042cdc5b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.562437 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be279e-e85b-47e5-af64-48042cdc5b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.563080 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be279e-e85b-47e5-af64-48042cdc5b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:59.564663 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be279e-e85b-47e5-af64-48042cdc5b81 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:59.574477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1440/2884] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:04:59.578179 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbd80c-f77d-4456-8e93-0a5cf10f64c2 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:59.588609 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbd80c-f77d-4456-8e93-0a5cf10f64c2 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:59.593152 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0d831-42b0-48da-bbd2-b2671226c60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:59.593771 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0d831-42b0-48da-bbd2-b2671226c60c admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.594055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0d831-42b0-48da-bbd2-b2671226c60c admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.594333 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0d831-42b0-48da-bbd2-b2671226c60c admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.594975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0d831-42b0-48da-bbd2-b2671226c60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:59.596589 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0d831-42b0-48da-bbd2-b2671226c60c admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:59.607479 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1445/2885] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:04:59.611501 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be279e-e85b-47e5-af64-48042cdc5b81 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:59.621238 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be279e-e85b-47e5-af64-48042cdc5b81 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:59.625823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05d93-6491-4f3a-9b36-6e00b8f26fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:59.626448 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05d93-6491-4f3a-9b36-6e00b8f26fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.626722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05d93-6491-4f3a-9b36-6e00b8f26fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.626991 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05d93-6491-4f3a-9b36-6e00b8f26fc0 admin admin] PATH_INFO: `/v3/projects/cf939187436d4978a7805ab5aa91005a/users/fba9bc42ba834887a018ea2b50fcf862/roles/47112770ffdc45638da090c45073888e` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.641047 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d05d93-6491-4f3a-9b36-6e00b8f26fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf939187436d4978a7805ab5aa91005a, user_id=fba9bc42ba834887a018ea2b50fcf862, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:59.644189 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d05d93-6491-4f3a-9b36-6e00b8f26fc0 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:59.657845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1441/2886] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:04:59 2022] PUT /identity/v3/projects/cf939187436d4978a7805ab5aa91005a/users/fba9bc42ba834887a018ea2b50fcf862/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:04:59.663454 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffadbc-a5e2-407a-ba04-b2aae2782d89 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.663817 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffadbc-a5e2-407a-ba04-b2aae2782d89 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.664198 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffadbc-a5e2-407a-ba04-b2aae2782d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.682750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-adffadbc-a5e2-407a-ba04-b2aae2782d89 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:04:59.683883 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-adffadbc-a5e2-407a-ba04-b2aae2782d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:59.741701 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1446/2887] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:04:59.751432 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05d93-6491-4f3a-9b36-6e00b8f26fc0 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:04:59.753108 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-a7d05d93-6491-4f3a-9b36-6e00b8f26fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:04:59.766917 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:04:59.810368 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05d93-6491-4f3a-9b36-6e00b8f26fc0 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:04:59.813947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72273103-807a-45cf-9e66-b59f8ea9cf13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:04:59.814540 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72273103-807a-45cf-9e66-b59f8ea9cf13 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:04:59.814816 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72273103-807a-45cf-9e66-b59f8ea9cf13 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:04:59.815085 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72273103-807a-45cf-9e66-b59f8ea9cf13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:04:59.824331 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72273103-807a-45cf-9e66-b59f8ea9cf13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:04:59.825800 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72273103-807a-45cf-9e66-b59f8ea9cf13 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:04:59.835853 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1442/2888] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:05:00.452784 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abadf7-2dbd-470e-ba9a-99a400630230 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:05:00.453271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abadf7-2dbd-470e-ba9a-99a400630230 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:05:00.453271 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abadf7-2dbd-470e-ba9a-99a400630230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:05:00.481226 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-f6abadf7-2dbd-470e-ba9a-99a400630230 None None] MFA Rules not processed for user `fba9bc42ba834887a018ea2b50fcf862`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:05:00.482491 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-f6abadf7-2dbd-470e-ba9a-99a400630230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:05:00.547751 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1447/2889] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:05:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:05:00.558716 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72273103-807a-45cf-9e66-b59f8ea9cf13 service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:05:00.560075 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-72273103-807a-45cf-9e66-b59f8ea9cf13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:05:00.617648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72273103-807a-45cf-9e66-b59f8ea9cf13 service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:05:00.621221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c1d0f-67ed-4dfd-972d-8e1953cdc456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a9421bec3b4b9e89dec35d7556d025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:05:00.621858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1d0f-67ed-4dfd-972d-8e1953cdc456 service cinder] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:05:00.622135 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1d0f-67ed-4dfd-972d-8e1953cdc456 service cinder] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:05:00.622404 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1d0f-67ed-4dfd-972d-8e1953cdc456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:05:00.631046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c1d0f-67ed-4dfd-972d-8e1953cdc456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:05:00.632489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c1d0f-67ed-4dfd-972d-8e1953cdc456 service cinder] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:05:00.641211 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1443/2890] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:05:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:05:02.007864 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715e6c7-f5d3-4c38-8014-f308cea5321a None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:05:02.008420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715e6c7-f5d3-4c38-8014-f308cea5321a None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:05:02.008420 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715e6c7-f5d3-4c38-8014-f308cea5321a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:05:02.029392 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6715e6c7-f5d3-4c38-8014-f308cea5321a None None] MFA Rules not processed for user `fba9bc42ba834887a018ea2b50fcf862`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:05:02.030604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6715e6c7-f5d3-4c38-8014-f308cea5321a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:05:02.040672 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1448/2891] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:05:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:05:02.275151 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c1d0f-67ed-4dfd-972d-8e1953cdc456 service cinder] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:05:02.284615 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c1d0f-67ed-4dfd-972d-8e1953cdc456 service cinder] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:05:02.288156 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ecdbf-3b11-4313-a532-fcc1970df4d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:05:02.288742 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ecdbf-3b11-4313-a532-fcc1970df4d1 service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:05:02.289019 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ecdbf-3b11-4313-a532-fcc1970df4d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:05:02.289288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ecdbf-3b11-4313-a532-fcc1970df4d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:05:02.291062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-6b6ecdbf-3b11-4313-a532-fcc1970df4d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:05:02.300845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ecdbf-3b11-4313-a532-fcc1970df4d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:05:02.302342 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ecdbf-3b11-4313-a532-fcc1970df4d1 service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:05:02.310293 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1444/2892] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:05:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:05:05.483970 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715e6c7-f5d3-4c38-8014-f308cea5321a None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:05:05.485410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6715e6c7-f5d3-4c38-8014-f308cea5321a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:05:05.491061 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:05:05.558471 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715e6c7-f5d3-4c38-8014-f308cea5321a None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:05:05.563665 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4772f-0340-44e0-8c6d-12ab0624842a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:05:05.564269 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4772f-0340-44e0-8c6d-12ab0624842a service placement] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:05:05.564544 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4772f-0340-44e0-8c6d-12ab0624842a service placement] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:05:05.564813 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4772f-0340-44e0-8c6d-12ab0624842a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:05:05.567123 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-7da4772f-0340-44e0-8c6d-12ab0624842a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:05:05.578837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4772f-0340-44e0-8c6d-12ab0624842a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:05:05.580397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4772f-0340-44e0-8c6d-12ab0624842a service placement] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:05:05.590387 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1449/2893] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:05:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:05:12.306760 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=52 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:05:15.586136 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=31 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:06:17.779483 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef361e9-d0b0-4170-9347-64c2e6d41fcd None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:06:17.780001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef361e9-d0b0-4170-9347-64c2e6d41fcd None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:06:17.780113 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef361e9-d0b0-4170-9347-64c2e6d41fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:06:17.798664 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-cef361e9-d0b0-4170-9347-64c2e6d41fcd None None] MFA Rules not processed for user `fba9bc42ba834887a018ea2b50fcf862`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:06:17.799862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-cef361e9-d0b0-4170-9347-64c2e6d41fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:06:17.806668 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1445/2894] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:06:27.793256 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=3 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:07:21.472380 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4772f-0340-44e0-8c6d-12ab0624842a service placement] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:21.483376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4772f-0340-44e0-8c6d-12ab0624842a service placement] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:21.488663 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ffd22-3587-4607-bfe4-9b451c97c62c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:21.489276 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ffd22-3587-4607-bfe4-9b451c97c62c service placement] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:21.489569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ffd22-3587-4607-bfe4-9b451c97c62c service placement] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:21.489867 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ffd22-3587-4607-bfe4-9b451c97c62c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:21.492278 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-812ffd22-3587-4607-bfe4-9b451c97c62c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:21.504354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ffd22-3587-4607-bfe4-9b451c97c62c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:21.505962 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ffd22-3587-4607-bfe4-9b451c97c62c service placement] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:21.517989 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1450/2895] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:07:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:07:21.531844 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef361e9-d0b0-4170-9347-64c2e6d41fcd None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:21.543537 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef361e9-d0b0-4170-9347-64c2e6d41fcd None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:21.548343 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:21.548911 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:21.549191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:21.549457 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] PATH_INFO: `/v3/users/fba9bc42ba834887a018ea2b50fcf862` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:21.550784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba9bc42ba834887a018ea2b50fcf862)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:21.552858 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:21.601366 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba9bc42ba834887a018ea2b50fcf862'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:07:21.602051 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba9bc42ba834887a018ea2b50fcf862'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:07:21.609020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba9bc42ba834887a018ea2b50fcf862'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:07:21.616555 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba9bc42ba834887a018ea2b50fcf862'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:07:21.617178 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-354c2b71-280b-4d5a-ab91-bfbd4bfb1a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba9bc42ba834887a018ea2b50fcf862'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:07:21.634122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1446/2896] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:07:21 2022] DELETE /identity/v3/users/fba9bc42ba834887a018ea2b50fcf862 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:07:21.806449 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ffd22-3587-4607-bfe4-9b451c97c62c service placement] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:21.817096 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ffd22-3587-4607-bfe4-9b451c97c62c service placement] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:21.873397 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:21.874051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:21.874327 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:21.874598 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] PATH_INFO: `/v3/projects/cf939187436d4978a7805ab5aa91005a` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:21.875693 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf939187436d4978a7805ab5aa91005a)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:21.877641 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:21.923087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf939187436d4978a7805ab5aa91005a', 'request_id': 'req-fa55063e-694b-49eb-8e71-e902a9fc67a3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:07:21.930043 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf939187436d4978a7805ab5aa91005a', 'request_id': 'req-fa55063e-694b-49eb-8e71-e902a9fc67a3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:07:21.936129 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf939187436d4978a7805ab5aa91005a', 'request_id': 'req-fa55063e-694b-49eb-8e71-e902a9fc67a3'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:07:21.946222 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] The token cache is being invalidate because project cf939187436d4978a7805ab5aa91005a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:07:21.946506 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:07:21.947120 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-fa55063e-694b-49eb-8e71-e902a9fc67a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:07:21.948538 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1451/2897] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:07:21 2022] DELETE /identity/v3/projects/cf939187436d4978a7805ab5aa91005a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:07:21.953390 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9f6fc-ff42-437f-998b-4b9608a8b363 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:21.953722 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9f6fc-ff42-437f-998b-4b9608a8b363 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:21.953978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9f6fc-ff42-437f-998b-4b9608a8b363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:21.974042 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-dea9f6fc-ff42-437f-998b-4b9608a8b363 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:07:21.975862 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-dea9f6fc-ff42-437f-998b-4b9608a8b363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:22.032543 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1447/2898] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:07:22.041851 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6277f0-d781-4b96-a120-7a44a7f224b6 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:22.042134 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6277f0-d781-4b96-a120-7a44a7f224b6 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:22.042453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6277f0-d781-4b96-a120-7a44a7f224b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:22.061535 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-0c6277f0-d781-4b96-a120-7a44a7f224b6 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:07:22.062664 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-0c6277f0-d781-4b96-a120-7a44a7f224b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:22.070659 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1452/2899] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:07:22.074734 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9f6fc-ff42-437f-998b-4b9608a8b363 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:22.084905 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9f6fc-ff42-437f-998b-4b9608a8b363 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:22.089901 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07175b6-9744-4fa4-a198-dda566c5774a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:22.090498 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07175b6-9744-4fa4-a198-dda566c5774a admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:22.090775 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07175b6-9744-4fa4-a198-dda566c5774a admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:22.091045 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07175b6-9744-4fa4-a198-dda566c5774a admin admin] PATH_INFO: `/v3/domains` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:22.091695 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07175b6-9744-4fa4-a198-dda566c5774a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87831) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:07:22.091994 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07175b6-9744-4fa4-a198-dda566c5774a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:22.093630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07175b6-9744-4fa4-a198-dda566c5774a admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:22.105978 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1448/2900] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 3 15:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:07:22.110126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6277f0-d781-4b96-a120-7a44a7f224b6 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:22.119119 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6277f0-d781-4b96-a120-7a44a7f224b6 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:22.123702 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0bd73-ff38-4cca-889c-ba5df088b5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:22.124287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0bd73-ff38-4cca-889c-ba5df088b5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:22.124567 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0bd73-ff38-4cca-889c-ba5df088b5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:22.124837 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0bd73-ff38-4cca-889c-ba5df088b5d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:22.125476 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0bd73-ff38-4cca-889c-ba5df088b5d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:22.127385 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0bd73-ff38-4cca-889c-ba5df088b5d9 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:22.154240 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1453/2901] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:07:22 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:07:22.158203 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07175b6-9744-4fa4-a198-dda566c5774a admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:22.168673 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07175b6-9744-4fa4-a198-dda566c5774a admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:22.223795 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3fa60-1152-4f17-8c2a-434fd1d3f13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:22.224399 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3fa60-1152-4f17-8c2a-434fd1d3f13f admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:22.224685 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3fa60-1152-4f17-8c2a-434fd1d3f13f admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:22.225559 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3fa60-1152-4f17-8c2a-434fd1d3f13f admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:22.226195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3fa60-1152-4f17-8c2a-434fd1d3f13f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:22.228155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3fa60-1152-4f17-8c2a-434fd1d3f13f admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:22.235965 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:07:22.249708 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1449/2902] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:07:22 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:07:22.253780 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0bd73-ff38-4cca-889c-ba5df088b5d9 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:22.264340 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0bd73-ff38-4cca-889c-ba5df088b5d9 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:22.268862 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38f06b-6953-43e8-bb4e-c3b9bb24559f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:22.269453 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38f06b-6953-43e8-bb4e-c3b9bb24559f admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:22.269761 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38f06b-6953-43e8-bb4e-c3b9bb24559f admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:22.270027 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38f06b-6953-43e8-bb4e-c3b9bb24559f admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:22.270667 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38f06b-6953-43e8-bb4e-c3b9bb24559f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:22.272884 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38f06b-6953-43e8-bb4e-c3b9bb24559f admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:22.282605 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1454/2903] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:07:22.286445 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3fa60-1152-4f17-8c2a-434fd1d3f13f admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:22.296409 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3fa60-1152-4f17-8c2a-434fd1d3f13f admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:22.301077 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e6b6a-0a2d-4264-9c2d-e348328170e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:22.301709 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e6b6a-0a2d-4264-9c2d-e348328170e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:22.301989 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e6b6a-0a2d-4264-9c2d-e348328170e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:22.302255 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e6b6a-0a2d-4264-9c2d-e348328170e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:22.302925 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e6b6a-0a2d-4264-9c2d-e348328170e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:22.304520 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e6b6a-0a2d-4264-9c2d-e348328170e4 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:22.315083 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1450/2904] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:07:22.319709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38f06b-6953-43e8-bb4e-c3b9bb24559f admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:22.329657 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38f06b-6953-43e8-bb4e-c3b9bb24559f admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:22.334184 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ac0ec-afaf-45f9-9a96-b2700ef31499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:22.335396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ac0ec-afaf-45f9-9a96-b2700ef31499 admin admin] REQUEST_METHOD: `PUT` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:22.335669 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ac0ec-afaf-45f9-9a96-b2700ef31499 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:22.335936 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ac0ec-afaf-45f9-9a96-b2700ef31499 admin admin] PATH_INFO: `/v3/projects/1b1db654fb9f4320ade1ca90e5000e09/users/12ee625591a14362a520aaad99db4ba8/roles/47112770ffdc45638da090c45073888e` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:22.350846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ac0ec-afaf-45f9-9a96-b2700ef31499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b1db654fb9f4320ade1ca90e5000e09, user_id=12ee625591a14362a520aaad99db4ba8, role_id=47112770ffdc45638da090c45073888e)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:22.353483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ac0ec-afaf-45f9-9a96-b2700ef31499 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:22.366083 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1455/2905] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:07:22 2022] PUT /identity/v3/projects/1b1db654fb9f4320ade1ca90e5000e09/users/12ee625591a14362a520aaad99db4ba8/roles/47112770ffdc45638da090c45073888e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:07:22.371031 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce24a2-5cb5-4f30-b0bd-ae2b91ef6af9 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:22.371317 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce24a2-5cb5-4f30-b0bd-ae2b91ef6af9 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:22.371587 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce24a2-5cb5-4f30-b0bd-ae2b91ef6af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:22.391660 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-1bce24a2-5cb5-4f30-b0bd-ae2b91ef6af9 None None] MFA Rules not processed for user `dd9397aad94548f284667a53c0b499d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:07:22.392856 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-1bce24a2-5cb5-4f30-b0bd-ae2b91ef6af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:22.449001 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1451/2906] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:07:22.458368 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ac0ec-afaf-45f9-9a96-b2700ef31499 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:22.460812 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-5b3ac0ec-afaf-45f9-9a96-b2700ef31499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:22.475126 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:07:22.518334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ac0ec-afaf-45f9-9a96-b2700ef31499 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:22.522562 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7d387-f0f8-4a2f-8240-af1f033777a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:22.523153 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7d387-f0f8-4a2f-8240-af1f033777a5 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:22.523431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7d387-f0f8-4a2f-8240-af1f033777a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:22.523704 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7d387-f0f8-4a2f-8240-af1f033777a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:22.532467 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7d387-f0f8-4a2f-8240-af1f033777a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:22.533922 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7d387-f0f8-4a2f-8240-af1f033777a5 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:22.542997 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1456/2907] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:07:26.504288 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaf2a9-a94b-44ab-9415-d0565bec263f None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:26.504823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaf2a9-a94b-44ab-9415-d0565bec263f None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:26.504823 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaf2a9-a94b-44ab-9415-d0565bec263f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:26.544369 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-aaeaf2a9-a94b-44ab-9415-d0565bec263f None None] MFA Rules not processed for user `12ee625591a14362a520aaad99db4ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:07:26.545706 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-aaeaf2a9-a94b-44ab-9415-d0565bec263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:26.636221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1452/2908] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:07:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:07:26.639652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7d387-f0f8-4a2f-8240-af1f033777a5 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:26.650827 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7d387-f0f8-4a2f-8240-af1f033777a5 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:26.657644 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb65ed-57db-4fa7-b8e3-ad931b64dc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:26.658423 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb65ed-57db-4fa7-b8e3-ad931b64dc17 admin admin] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:26.658709 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb65ed-57db-4fa7-b8e3-ad931b64dc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:26.659687 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb65ed-57db-4fa7-b8e3-ad931b64dc17 admin admin] PATH_INFO: `/v3/projects` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:26.661022 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb65ed-57db-4fa7-b8e3-ad931b64dc17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:26.663170 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb65ed-57db-4fa7-b8e3-ad931b64dc17 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:26.705944 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1457/2909] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 3 15:07:26 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:07:26.709313 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaf2a9-a94b-44ab-9415-d0565bec263f None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:26.730962 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaf2a9-a94b-44ab-9415-d0565bec263f None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:26.793826 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f207a7-e222-4f37-9996-2cfcade83137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:26.794512 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f207a7-e222-4f37-9996-2cfcade83137 admin admin] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:26.794904 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f207a7-e222-4f37-9996-2cfcade83137 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:26.795179 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f207a7-e222-4f37-9996-2cfcade83137 admin admin] PATH_INFO: `/v3/users` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:26.795947 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f207a7-e222-4f37-9996-2cfcade83137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:26.798466 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f207a7-e222-4f37-9996-2cfcade83137 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:26.820902 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:07:26.829253 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1453/2910] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 3 15:07:26 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:07:26.833891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb65ed-57db-4fa7-b8e3-ad931b64dc17 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:26.846430 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb65ed-57db-4fa7-b8e3-ad931b64dc17 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:26.852735 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81634155-c425-4f7a-8227-d4b5013be254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:26.853602 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81634155-c425-4f7a-8227-d4b5013be254 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:26.853949 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81634155-c425-4f7a-8227-d4b5013be254 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:26.854283 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81634155-c425-4f7a-8227-d4b5013be254 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:26.859475 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81634155-c425-4f7a-8227-d4b5013be254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:26.861224 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81634155-c425-4f7a-8227-d4b5013be254 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:26.874788 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1458/2911] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:07:26.878499 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f207a7-e222-4f37-9996-2cfcade83137 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:26.889474 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f207a7-e222-4f37-9996-2cfcade83137 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:26.894320 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9122c1-d355-45dc-83fc-cfec3d167119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:26.894999 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9122c1-d355-45dc-83fc-cfec3d167119 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:26.895278 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9122c1-d355-45dc-83fc-cfec3d167119 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:26.895577 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9122c1-d355-45dc-83fc-cfec3d167119 admin admin] PATH_INFO: `/v3/projects/5b98d30303b5404c9c4ad3298881e55c/users/cc40fffda2e54283aa26d57e4d49cbe4/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:26.912033 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9122c1-d355-45dc-83fc-cfec3d167119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b98d30303b5404c9c4ad3298881e55c, user_id=cc40fffda2e54283aa26d57e4d49cbe4, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:26.915294 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9122c1-d355-45dc-83fc-cfec3d167119 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:26.934116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1454/2912] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:07:26 2022] PUT /identity/v3/projects/5b98d30303b5404c9c4ad3298881e55c/users/cc40fffda2e54283aa26d57e4d49cbe4/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:07:26.938356 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81634155-c425-4f7a-8227-d4b5013be254 admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:26.948497 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81634155-c425-4f7a-8227-d4b5013be254 admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:27.007802 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d75da-9a3e-4fa4-b284-e83763abc408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:27.008417 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d75da-9a3e-4fa4-b284-e83763abc408 admin admin] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:27.008703 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d75da-9a3e-4fa4-b284-e83763abc408 admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:27.008980 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d75da-9a3e-4fa4-b284-e83763abc408 admin admin] PATH_INFO: `/v3/roles` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:27.009716 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d75da-9a3e-4fa4-b284-e83763abc408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:27.011419 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d75da-9a3e-4fa4-b284-e83763abc408 admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:27.022410 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1459/2913] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 3 15:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:07:27.025653 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9122c1-d355-45dc-83fc-cfec3d167119 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:27.035981 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9122c1-d355-45dc-83fc-cfec3d167119 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:27.041084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be985f9-4d27-4a5b-8fa8-cfd89feb7fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:27.041753 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be985f9-4d27-4a5b-8fa8-cfd89feb7fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:27.042030 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be985f9-4d27-4a5b-8fa8-cfd89feb7fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:27.042296 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be985f9-4d27-4a5b-8fa8-cfd89feb7fb2 admin admin] PATH_INFO: `/v3/projects/5b98d30303b5404c9c4ad3298881e55c/users/cc40fffda2e54283aa26d57e4d49cbe4/roles/ea657fbc11e2449b943418b35bf4a4ff` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:27.044202 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be985f9-4d27-4a5b-8fa8-cfd89feb7fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b98d30303b5404c9c4ad3298881e55c, user_id=cc40fffda2e54283aa26d57e4d49cbe4, role_id=ea657fbc11e2449b943418b35bf4a4ff)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:27.046975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be985f9-4d27-4a5b-8fa8-cfd89feb7fb2 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:27.057758 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1455/2914] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 3 15:07:27 2022] PUT /identity/v3/projects/5b98d30303b5404c9c4ad3298881e55c/users/cc40fffda2e54283aa26d57e4d49cbe4/roles/ea657fbc11e2449b943418b35bf4a4ff => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:07:31.431473 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bf868-2ae6-47de-830c-e18f342b9f16 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:31.432002 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bf868-2ae6-47de-830c-e18f342b9f16 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:31.432111 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bf868-2ae6-47de-830c-e18f342b9f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:31.469741 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-726bf868-2ae6-47de-830c-e18f342b9f16 None None] MFA Rules not processed for user `cc40fffda2e54283aa26d57e4d49cbe4`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:07:31.470978 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-726bf868-2ae6-47de-830c-e18f342b9f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:31.541395 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1460/2915] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:07:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:07:31.554402 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be985f9-4d27-4a5b-8fa8-cfd89feb7fb2 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:31.557521 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-4be985f9-4d27-4a5b-8fa8-cfd89feb7fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:31.649832 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be985f9-4d27-4a5b-8fa8-cfd89feb7fb2 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:31.654505 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dfd52-9225-4b74-a93e-681659d9822c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd04df68f9d84fc4bb981c8b69cc771a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:31.655194 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dfd52-9225-4b74-a93e-681659d9822c service glance] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:31.655539 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dfd52-9225-4b74-a93e-681659d9822c service glance] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:31.655878 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dfd52-9225-4b74-a93e-681659d9822c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:31.727452 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915dfd52-9225-4b74-a93e-681659d9822c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:31.728891 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915dfd52-9225-4b74-a93e-681659d9822c service glance] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:31.738006 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1456/2916] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:07:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:07:31.868225 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be56d1-11db-464f-94f2-f91b72a891a0 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:31.868533 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be56d1-11db-464f-94f2-f91b72a891a0 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:31.868807 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be56d1-11db-464f-94f2-f91b72a891a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:31.876983 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:07:31.889489 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-c6be56d1-11db-464f-94f2-f91b72a891a0 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:07:31.890732 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-c6be56d1-11db-464f-94f2-f91b72a891a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:31.950975 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1461/2917] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 15:07:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:07:31.996563 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dfd52-9225-4b74-a93e-681659d9822c service glance] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:31.998020 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-915dfd52-9225-4b74-a93e-681659d9822c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:32.058295 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:07:32.074082 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dfd52-9225-4b74-a93e-681659d9822c service glance] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:32.078990 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ddef3-b409-4ed2-9127-c83185a991d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:32.079601 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ddef3-b409-4ed2-9127-c83185a991d0 service swift] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:32.079866 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ddef3-b409-4ed2-9127-c83185a991d0 service swift] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:32.080131 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ddef3-b409-4ed2-9127-c83185a991d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:32.087957 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ddef3-b409-4ed2-9127-c83185a991d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:32.089825 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ddef3-b409-4ed2-9127-c83185a991d0 service swift] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:32.097656 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1457/2918] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 15:07:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:07:40.705824 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e490c-3a8d-4ec9-907a-73712941f437 None None] REQUEST_METHOD: `POST` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:40.706360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e490c-3a8d-4ec9-907a-73712941f437 None None] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:40.706360 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e490c-3a8d-4ec9-907a-73712941f437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:40.725274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.auth.core [None req-6a6e490c-3a8d-4ec9-907a-73712941f437 None None] MFA Rules not processed for user `12ee625591a14362a520aaad99db4ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=87830) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:07:40.726465 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6a6e490c-3a8d-4ec9-907a-73712941f437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:40.733541 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1462/2919] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 3 15:07:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:07:40.773944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ddef3-b409-4ed2-9127-c83185a991d0 service swift] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:40.783115 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ddef3-b409-4ed2-9127-c83185a991d0 service swift] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:40.786672 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d46a4-5042-4736-b7c0-a7cb5e261de9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd04df68f9d84fc4bb981c8b69cc771a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:40.787259 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d46a4-5042-4736-b7c0-a7cb5e261de9 service glance] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:40.787533 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d46a4-5042-4736-b7c0-a7cb5e261de9 service glance] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:40.787797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d46a4-5042-4736-b7c0-a7cb5e261de9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:40.789479 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-8e9d46a4-5042-4736-b7c0-a7cb5e261de9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:40.858110 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d46a4-5042-4736-b7c0-a7cb5e261de9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:40.859472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d46a4-5042-4736-b7c0-a7cb5e261de9 service glance] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:40.870843 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1458/2920] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:07:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:07:41.404542 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e490c-3a8d-4ec9-907a-73712941f437 None None] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:41.406026 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-6a6e490c-3a8d-4ec9-907a-73712941f437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:41.463891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e490c-3a8d-4ec9-907a-73712941f437 None None] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:41.467556 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e197e-e580-4eae-82ee-f6cfbdb5dfa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:41.468151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e197e-e580-4eae-82ee-f6cfbdb5dfa1 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:41.468431 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e197e-e580-4eae-82ee-f6cfbdb5dfa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:41.468713 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e197e-e580-4eae-82ee-f6cfbdb5dfa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:41.470566 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-d56e197e-e580-4eae-82ee-f6cfbdb5dfa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:41.485124 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e197e-e580-4eae-82ee-f6cfbdb5dfa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:41.486660 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e197e-e580-4eae-82ee-f6cfbdb5dfa1 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:41.498396 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1463/2921] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:07:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:07:42.689472 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f5fff-6fe5-463e-b6c0-c92affa57ef2 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:42.689996 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f5fff-6fe5-463e-b6c0-c92affa57ef2 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:42.690122 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f5fff-6fe5-463e-b6c0-c92affa57ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:42.709640 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-eb0f5fff-6fe5-463e-b6c0-c92affa57ef2 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:07:42.710845 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-eb0f5fff-6fe5-463e-b6c0-c92affa57ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:07:42.718249 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1459/2922] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 3 15:07:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:07:42.731438 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e197e-e580-4eae-82ee-f6cfbdb5dfa1 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:07:42.743457 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e197e-e580-4eae-82ee-f6cfbdb5dfa1 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:07:42.748483 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea4f44-ed32-4235-b00f-959ec6c9c457 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:07:42.749056 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea4f44-ed32-4235-b00f-959ec6c9c457 service swift] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:07:42.749334 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea4f44-ed32-4235-b00f-959ec6c9c457 service swift] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:07:42.749662 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea4f44-ed32-4235-b00f-959ec6c9c457 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:07:42.758139 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea4f44-ed32-4235-b00f-959ec6c9c457 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:07:42.759921 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea4f44-ed32-4235-b00f-959ec6c9c457 service swift] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:07:42.768293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1464/2923] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 15:07:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:07:52.703977 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=44 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:07:52.764846 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=58 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:08:13.122148 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f5fff-6fe5-463e-b6c0-c92affa57ef2 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:08:13.132129 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f5fff-6fe5-463e-b6c0-c92affa57ef2 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:08:13.135627 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41096484-4bdf-41fc-84e4-9b3695b40b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:08:13.136216 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41096484-4bdf-41fc-84e4-9b3695b40b8d service neutron] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:08:13.136489 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41096484-4bdf-41fc-84e4-9b3695b40b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:08:13.136784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41096484-4bdf-41fc-84e4-9b3695b40b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:08:13.145358 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41096484-4bdf-41fc-84e4-9b3695b40b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:08:13.147062 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41096484-4bdf-41fc-84e4-9b3695b40b8d service neutron] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:08:13.156221 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1460/2924] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:08:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:08:15.905164 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea4f44-ed32-4235-b00f-959ec6c9c457 service swift] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:08:15.914782 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea4f44-ed32-4235-b00f-959ec6c9c457 service swift] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:08:15.918390 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfea53-2224-4fac-821b-ca863aee4132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e9009410a54ce3ab88fa0cac87a7f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:08:15.919009 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfea53-2224-4fac-821b-ca863aee4132 service neutron] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:08:15.919292 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfea53-2224-4fac-821b-ca863aee4132 service neutron] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:08:15.919565 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfea53-2224-4fac-821b-ca863aee4132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:08:15.921287 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-12bfea53-2224-4fac-821b-ca863aee4132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:08:15.931629 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfea53-2224-4fac-821b-ca863aee4132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:08:15.933051 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfea53-2224-4fac-821b-ca863aee4132 service neutron] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:08:15.942057 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1465/2925] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:08:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:08:23.152195 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=6 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:08:25.938151 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=6 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:10:12.576128 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41096484-4bdf-41fc-84e4-9b3695b40b8d service neutron] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:10:12.577564 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-41096484-4bdf-41fc-84e4-9b3695b40b8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:10:12.712691 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41096484-4bdf-41fc-84e4-9b3695b40b8d service neutron] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:10:12.718469 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102d1a9-35aa-46f0-91d7-e959a9ec4815 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:10:12.719069 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102d1a9-35aa-46f0-91d7-e959a9ec4815 service placement] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:10:12.719347 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102d1a9-35aa-46f0-91d7-e959a9ec4815 service placement] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:10:12.719619 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102d1a9-35aa-46f0-91d7-e959a9ec4815 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:10:12.721299 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7102d1a9-35aa-46f0-91d7-e959a9ec4815 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:10:12.733032 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102d1a9-35aa-46f0-91d7-e959a9ec4815 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:10:12.734630 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102d1a9-35aa-46f0-91d7-e959a9ec4815 service placement] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:10:12.744269 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1461/2926] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:10:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:10:22.740084 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=63 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:10:32.991748 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfea53-2224-4fac-821b-ca863aee4132 service neutron] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:10:32.999087 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-12bfea53-2224-4fac-821b-ca863aee4132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:10:33.059621 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfea53-2224-4fac-821b-ca863aee4132 service neutron] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:10:33.066909 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f778a16-f02e-4780-8f02-eba46c47f91c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a4a95ae5ee84f259ffcd7528b0fa505', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:10:33.068112 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f778a16-f02e-4780-8f02-eba46c47f91c service nova] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:10:33.068638 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f778a16-f02e-4780-8f02-eba46c47f91c service nova] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:10:33.069078 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f778a16-f02e-4780-8f02-eba46c47f91c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:10:33.072536 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-4f778a16-f02e-4780-8f02-eba46c47f91c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:10:33.092569 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f778a16-f02e-4780-8f02-eba46c47f91c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:10:33.094223 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f778a16-f02e-4780-8f02-eba46c47f91c service nova] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:10:33.106274 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1466/2927] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:10:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:10:40.474130 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac2c63-f4fa-418c-b3d1-a0a1795dd0f0 None None] REQUEST_METHOD: `POST` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:10:40.474583 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac2c63-f4fa-418c-b3d1-a0a1795dd0f0 None None] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:10:40.474693 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac2c63-f4fa-418c-b3d1-a0a1795dd0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:10:40.494116 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.auth.core [None req-d1ac2c63-f4fa-418c-b3d1-a0a1795dd0f0 None None] MFA Rules not processed for user `fd0db9d2a753490cb8473229a8f6fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=87831) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:10:40.495312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-d1ac2c63-f4fa-418c-b3d1-a0a1795dd0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:10:40.502975 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1462/2928] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 3 15:10:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:10:40.541937 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f778a16-f02e-4780-8f02-eba46c47f91c service nova] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:10:40.553253 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f778a16-f02e-4780-8f02-eba46c47f91c service nova] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:10:40.558760 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef88154-c25b-4bc3-bcc1-4af846fe520f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbf23ae20ee4f6c979e256d2d4daa5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:10:40.559354 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef88154-c25b-4bc3-bcc1-4af846fe520f service swift] REQUEST_METHOD: `GET` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:10:40.559640 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef88154-c25b-4bc3-bcc1-4af846fe520f service swift] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:10:40.559914 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef88154-c25b-4bc3-bcc1-4af846fe520f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:10:40.567677 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef88154-c25b-4bc3-bcc1-4af846fe520f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:10:40.569293 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef88154-c25b-4bc3-bcc1-4af846fe520f service swift] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:10:40.577787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1467/2929] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 3 15:10:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:10:46.865576 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac2c63-f4fa-418c-b3d1-a0a1795dd0f0 None None] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:10:46.950852 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac2c63-f4fa-418c-b3d1-a0a1795dd0f0 None None] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:10:46.955638 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:10:46.956309 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:10:46.956595 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:10:46.956875 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] PATH_INFO: `/v3/users/12ee625591a14362a520aaad99db4ba8` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:10:46.958046 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12ee625591a14362a520aaad99db4ba8)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:10:46.960093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:10:47.011633 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12ee625591a14362a520aaad99db4ba8'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.012302 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12ee625591a14362a520aaad99db4ba8'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.020013 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12ee625591a14362a520aaad99db4ba8'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.031056 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12ee625591a14362a520aaad99db4ba8'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.031676 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12ee625591a14362a520aaad99db4ba8'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.049433 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1463/2930] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:10:46 2022] DELETE /identity/v3/users/12ee625591a14362a520aaad99db4ba8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:10:47.223620 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef88154-c25b-4bc3-bcc1-4af846fe520f service swift] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:10:47.239055 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef88154-c25b-4bc3-bcc1-4af846fe520f service swift] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:10:47.296099 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:10:47.296634 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:10:47.296917 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:10:47.297195 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] PATH_INFO: `/v3/projects/1b1db654fb9f4320ade1ca90e5000e09` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:10:47.300787 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b1db654fb9f4320ade1ca90e5000e09)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:10:47.302752 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:10:47.348517 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b1db654fb9f4320ade1ca90e5000e09', 'request_id': 'req-3ac1881b-5391-4dd1-97e8-324cfce29fed'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.356154 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b1db654fb9f4320ade1ca90e5000e09', 'request_id': 'req-3ac1881b-5391-4dd1-97e8-324cfce29fed'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.362258 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b1db654fb9f4320ade1ca90e5000e09', 'request_id': 'req-3ac1881b-5391-4dd1-97e8-324cfce29fed'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.384338 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] The token cache is being invalidate because project 1b1db654fb9f4320ade1ca90e5000e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:10:47.384624 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.385236 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.386699 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1468/2931] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:10:47 2022] DELETE /identity/v3/projects/1b1db654fb9f4320ade1ca90e5000e09 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:10:47.390648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:10:47.392093 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:10:47.406260 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3481679-0cef-4b2e-96ee-73f285149394 admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:10:47.455751 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:10:47.463155 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:10:47.463784 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:10:47.464477 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:10:47.464797 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] PATH_INFO: `/v3/users/cc40fffda2e54283aa26d57e4d49cbe4` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:10:47.466312 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc40fffda2e54283aa26d57e4d49cbe4)` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:10:47.468714 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:10:47.514431 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc40fffda2e54283aa26d57e4d49cbe4'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.515117 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc40fffda2e54283aa26d57e4d49cbe4'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.521963 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc40fffda2e54283aa26d57e4d49cbe4'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.529289 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc40fffda2e54283aa26d57e4d49cbe4'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.529944 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.notifications [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc40fffda2e54283aa26d57e4d49cbe4'} {{(pid=87831) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.547247 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1464/2932] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 3 15:10:47 2022] DELETE /identity/v3/users/cc40fffda2e54283aa26d57e4d49cbe4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:10:47.718583 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] Authenticating user token {{(pid=87830) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:10:47.720074 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.fernet_utils [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87830) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:10:47.734998 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1881b-5391-4dd1-97e8-324cfce29fed admin admin] Validating token access rules against request {{(pid=87830) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:10:47.757116 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:10:47.795750 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9397aad94548f284667a53c0b499d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47484d92809b4b9a846447c512be0a94', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87830) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:10:47.796376 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:10:47.796655 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:10:47.796926 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] PATH_INFO: `/v3/projects/5b98d30303b5404c9c4ad3298881e55c` {{(pid=87830) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:10:47.798156 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b98d30303b5404c9c4ad3298881e55c)` {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:10:47.799996 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] RBAC: Authorization granted {{(pid=87830) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:10:47.846266 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b98d30303b5404c9c4ad3298881e55c', 'request_id': 'req-145fc3b2-bc6d-4063-b968-76158dd2fb8d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.853114 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b98d30303b5404c9c4ad3298881e55c', 'request_id': 'req-145fc3b2-bc6d-4063-b968-76158dd2fb8d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.859682 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b98d30303b5404c9c4ad3298881e55c', 'request_id': 'req-145fc3b2-bc6d-4063-b968-76158dd2fb8d'} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.869652 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] The token cache is being invalidate because project 5b98d30303b5404c9c4ad3298881e55c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87830) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:10:47.869941 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.871161 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG keystone.notifications [None req-145fc3b2-bc6d-4063-b968-76158dd2fb8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87830) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:10:47.872604 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: [pid: 87830|app: 0|req: 1469/2933] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 3 15:10:47 2022] DELETE /identity/v3/projects/5b98d30303b5404c9c4ad3298881e55c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:10:57.543757 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=28,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:10:57.866891 dv-cinder-859877-2-iscsi devstack@keystone.service[87830]: DEBUG dbcounter [-] [87830] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87830) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:15:16.498806 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] Authenticating user token {{(pid=87831) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:15:16.500191 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:15:16.568143 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e23e7f-cf3b-4880-94c5-d60646f1985e admin admin] Validating token access rules against request {{(pid=87831) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:15:16.625624 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d767e6d-e868-4743-a140-24c6ac9eed97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc01c87551044818e1ccbb46b8e7340', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a912636f54fc41d0b7fa1a150154b06a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87831) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:15:16.626233 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d767e6d-e868-4743-a140-24c6ac9eed97 service placement] REQUEST_METHOD: `GET` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:15:16.626641 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d767e6d-e868-4743-a140-24c6ac9eed97 service placement] SCRIPT_NAME: `/identity` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:15:16.626945 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d767e6d-e868-4743-a140-24c6ac9eed97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87831) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:15:16.628648 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.fernet_utils [None req-7d767e6d-e868-4743-a140-24c6ac9eed97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87831) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:15:16.643107 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d767e6d-e868-4743-a140-24c6ac9eed97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:15:16.645209 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d767e6d-e868-4743-a140-24c6ac9eed97 service placement] RBAC: Authorization granted {{(pid=87831) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:15:16.656061 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: [pid: 87831|app: 0|req: 1465/2934] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 3 15:15:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:15:26.651651 dv-cinder-859877-2-iscsi devstack@keystone.service[87831]: DEBUG dbcounter [-] [87831] Writing DB stats keystone:SELECT=58 {{(pid=87831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}